Controlling trains with an iPhone / iPod Touch

[serenityFan]

Also, I generally just want to get to know Objective C and Cocoa and program for iPhone / iPod Touch. I think I have some interesting ideas with regards to that, and have already started working on a prototype app.

Wait a sec, are you saying we are going to see an iphone app that can control our trains? Made by you? Cool .....   

[Martijn Meerts]

Also, I generally just want to get to know Objective C and Cocoa and program for iPhone / iPod Touch. I think I have some interesting ideas with regards to that, and have already started working on a prototype app.

Wait a sec, are you saying we are going to see an iphone app that can control our trains? Made by you? Cool .....   

Possibly, but it won't be as easy as installing the app and running. There'll need to be a program on a computer which the layout is connected to, which sends on the commands received from the iphone.

I have a bunch of idea, but I don't really want to throw them out in the open just yet. But if anyone is interested, feel free to send me a PM for some more detailed discussions ;)


There is actually an app already to control trains, but you need either a Marklin Central Station or the ESU ECoS.

[Martijn Meerts]

Split this topic from the original one so we can talk a bit about it...

I've been progressing a bit faster than I thought I would, I don't have anything that can actually control a train yet, but the iPhone / iPod Touch bit is coming along nicely.


The initial idea was to build a Mac OS X app that would combine a model train database, decoder programmer, and manual throttle control (so, for now at least, no blocks, occupancy detection, signals, turnouts etc.) I actually started working on this a while ago, and have 1 of the more challenging technical bits working. I can actually send commands to the command station via the interface, and the trains will respond to them. Since my Japanese trains use a Lenz LI-USB interface, that's so far the only one that works. I also still have the commands hard coded in the program, so it's very limited, but the connection is there ;)

Looking a bit further at interface elements and database storage and the likes, I quickly found out that the OS X API has an insane amount of frameworks, and each of them have an insane amount of functionality. So, I decided to first look at iPhone development, because obviously it's much more limited.

On the iPhone, I've managed to clear a few initial technical hurdles. I can test the availability of WiFi, and I've managed to set up a basic database so that trains can be stored locally on the iPhone.

To control trains using the iPhone app, it requires a pairing with the computer that the interface is connected to. So, the OS X app will also include a server that the iPhone app can connect to. For those who have an iPhone / iPod Touch, install Apple's "Remote" app, which allows you to control your iTunes library via the iPhone. That uses the same mechanism I was thinking of using. I've come a ways there as well, I can browse the WiFi network for machines that are running a certain service, and then connect to that machine.


My main issue right now is actually the interface. While it's easy enough to create an interface to pick a train to control, I'm really rather stuck with how the actual "throttle" should look. I also have only a very basic idea of how I want the OS X app to look. I may be able to design webpages, but application interfaces is definitely not my thing =)

Ideas and concept sketches would be greatly appreciated ;)

[alpineaustralia]

You lost me just after "Split this topic from the original one so we can talk a bit about it..."
Sounds exciting mate.
At thisstage I have just recived my first ipod nano and am coming to grips with it.

[CaptOblivious]

Martijn,

I'm about to drop the dough for an Android phone. Would be interesting to do a tandem iPhone/Android release :D I would be willing to chip in time and effort on that!

[to2leo]

I think we should discard the whole DCC thing and just take advantage of the RC technology.

[Martijn Meerts]

Martijn,

I'm about to drop the dough for an Android phone. Would be interesting to do a tandem iPhone/Android release :D I would be willing to chip in time and effort on that!

It should be possible to hook up an Android phone, as long as the Android phone sends and receives the same data from the "server" as the iPhone app.

Networking on the iPhone (and OS X for that matter) isn't all that easy. Detecting servers/clients works fine, you can just use Bonjour for that. However, creating sockets and sending/receiving info tends to require some low-level C, something I'm not comfortable with.

I'm probably going to use http://bitbucket.org/snej/mynetwork/wiki/Home as the basis of pairing the iPhone app and OS X app.

It would of course also be possible to write a windows server app, but we'd need someone with windows programming knowledge =)

I think we should discard the whole DCC thing and just take advantage of the RC technology.

Hmm.. Not sure I can see a lot of advantages of RC compared to DCC, especially not once you start with computer control and automating layouts ;)

[disturbman]

Interesting thread.

Shouldn't the throttle be a "horizontal slide buton"? You could have it graduated to have an idea of the "speed" (from 0 to 100 in percentage) and a window displaying the precise value in digital number.

[Martijn Meerts]

I'm pretty sure that a slider of sorts is the best to select the speed. Not sure if horizontal or vertical is better from a usability stand point. A vertical one can be bigger, which means better control.

But that's only the speed of the train, the "throttle" (maybe it's better to call it controller really) also needs buttons for functions and indicators for direction of travel and the likes. The question then is if it would be better to simulate a "real" controller, or make it much more abstract with standardized buttons etc.

Apple's Human Interface Guidelines don't help much here, since they state that if you make a virtual version of a real device, it might be best to make it look like the real device, so that people will instantly know how to use it. On the other hand, it also states you shouldn't make custom buttons and controls unless it's really necessary =)

[disturbman]

My advice is: it must be easy to use and handle. That's the key feature and the design must reflect it.

As for the size and direction of the controller, with the Iphone the screen can rotate so you can use this feature at your advantage by using the horizontal control in the greater dimension. You will also have to test (or ask around) to see if it's easier to move your hand on the horizontal or in the vertical. For me that's clear, it's the horizon.

[Shashinka]

Oooh, ooh, I have a shuffle, I can't wait to see how my trains would run with it.

[serenityFan]

So basically you would need to be able to control your layout from a computer, and then the iphone simply sends instructions to the computer which controls the layout? I better learn how to do the first part first 

Martjin, what about available modules, like JRMI? Would that be incompatible with what you are building?

[Martijn Meerts]

So basically you would need to be able to control your layout from a computer, and then the iphone simply sends instructions to the computer which controls the layout? I better learn how to do the first part first 

Martjin, what about available modules, like JRMI? Would that be incompatible with what you are building?

If you have a Marklin CS / CS2 or an ESU ECoS, you would be able to skip the computer part, and have the iPhone send commands directly to the command station. The reason for that is that those command stations have a network connection. Pretty much all other command stations are either USB or good 'ol RS232.

Of course, the advantage of requiring a computer, is that it's possible to support all digital systems, even those not released yet.


It would be possible to use JRMI, and build an iPhone add-on on top of that, but I don't know Java at all, and I don't feel like learning both Java and Objective-C (Objective-C I need for iPhone development)

[Shashinka]

Oooh, ooh, I have a shuffle, I can't wait to see how my trains would run with it.

...tumbleweeds roll on by.

[Martijn Meerts]

Oooh, ooh, I have a shuffle, I can't wait to see how my trains would run with it.

...tumbleweeds roll on by.

They won't because you can't install apps on the shuffle ;)

[cteno4]

Martijn,

I find this thread very intriguing! would be very cool controller. what about just networking into a netbook running jrmi and sending jrmi commands (sure they must have some sort of external communication protocol) and letting it do the running/control of the dcc stuff? would mean you just have to learn how to tell jrmi to accelerate, break, turn on/off what decoder and then let it do the actual dcc work/communication.

rub maybe having a tcpip protocol with jrmi to talk to it, the world is slowly going that way. most of the equipment though that is not native tcpip is using telnet to make the bridge and thats a royal pain in the patute to keep open all the time!

im just starting to really look at the touch for exhibit purposes, thing have finally gotten to a really great nexus of functionality, design, software apis, cost, etc to make it a very viable option for visitor use!

cheers

jeff

[CaptOblivious]

I'm also starting to toy with the idea of designing a LocoNet/Lenz XPressNet -> TCP/IP device using ARduino. Would allow the command station to be connected directly to your LAN (or the internet!), again bypassing the need for an intermediary computer just to control a train. That could be fun.

[Martijn Meerts]

Martijn,

I find this thread very intriguing! would be very cool controller. what about just networking into a netbook running jrmi and sending jrmi commands (sure they must have some sort of external communication protocol) and letting it do the running/control of the dcc stuff? would mean you just have to learn how to tell jrmi to accelerate, break, turn on/off what decoder and then let it do the actual dcc work/communication.

That would still require extensive Java knowledge, since I don't think JRMI accepts bi-directional connections from external devices over wifi. I also have no idea how hard that is to implement in JRMI. Don might be able to say more about this, I believe he knows a bit of Java ;)


Another reason I wanted to start from scratch, is that I have a couple of ideas floating around, which would be much easier when having a custom OS X app compared to modifying / adding to JRMI. One of those things (which might be interesting for you with regards to exhibitions), is to allow the OS X app to limit which train(s) are possible to be controlled with each paired iPhone / iPod Touch, and limit certain settings (max speed etc.) That basically means you could have an iPod Touch available for visitors, who can then control a train on the layout. Pretty sure kids would love to be able to control Doraemon or Dr. Yellow =)

Another idea is to have the OS X app feed data to an iPod touch which you have mounted somewhere near a large station. Based on some block detection system, the OS X app would know which trains are about to enter the station, and can tell the iPod Touch to display which train will arrive on which track etc.

[cteno4]

Martijn,

sorry i have not take the time to really look into the jrmi system. was hoping that they were making a very open system to allow easy communication with external elements like this. i too have not taken the time to really dive deeply into java, one of those things that you have to have a big need to feed the learning curve!

these would be wonderful for show layouts for both member and visitor control of trains. dcc makes it perfect to limit speeds and actions so visitors cant do horrible things. we have been looking at dcc to make a visitor throttle and just wire the decoder to the tracks so we can use non dcc trains, but still limit the throttle speeds and acel and decel.

only problem with touches at a train show with visitor use is they could walk off easily!

btw, im actually looking into the use of the touchs in museum exhibits to deliver content in interesting and different ways. it allows you to keep stuff up on the walls to the big picture and getting folks interested, then followup with more detailed and varied presentations on the touch that folks can dive into (and down in multiple levels) depending upon what they are most interested in. this is traditionally done in computer kiosks, but those are physically imposing, expensive and hard to design into your exhibit flow. for one kiosk you could buy like 25 touchs, so lots more folks could interact with the at once and at whatever place and time in the space they want to. main downside is that they can be isolating, especially if you do audio, so this is the design challenge to see if it can be done w/o isolating visitors from each other and maybe even figure out ways of using it to get visitors to interact with each other in novel ways in the exhibit space.

cheers

jeff

[Martijn Meerts]

I haven't looked much into JRMI either to be honest, I only use if for decoder programming. For anything else I just think it's too confusing ;)

As for walking of with an iPod Touch, that could be a problem of course. However, there are 2 main ways this could be solved. If you supply your own iPod Touch, you could basically hook up some security wire to it. It'll make things a little more limiting, but it's better than having to buy new iPods for each show. Another option is to allow visitors who have their own iPods to pair their iPod with the OS X app. As the admin, you could then specify which train that iPod gets access to. Of course, they'd have to install the app, but the idea is to make it available for free anyway ;)


What could be interesting for a museum would be to use a form of Augmented Reality. Unfortunately, it requires a camera, so it won't work with the current iPod Touches, but it would allow you to overlay an interface onto the camera image. If you could somehow make it so the iPod would recognize the exhibition pieces, you could then tap the piece on the screen of the iPod, and more info would pop up.

<a href="http://www.youtube.com/v/_0bitKDKdg0" target="_blank">http://www.youtube.com/v/_0bitKDKdg0</a>

This is a good example of recognizing and tracking markers. It's all a bit slow, but it's still early development ;)

[Martijn Meerts]

Here's a screen of what I've been able to do so far. All these screenshots are taken from an actual device (as I'm a registered iPhone developer, I can test my stuff on a device, not just on the simulator =))

Left to right, top to bottom:

1 - Initial screen, it checks to see if WiFi is available (required to connect to the OS X app, iPhone app still usable as database when WiFi is off)
2 - List of trains, showing "name" and "description", still need to add thumbnail image. This data is taken from a database, and it not static.
3 - Select a train and...
4 - ... you get the "throttle view", although, there's not much of a throttle there yet ;)

5 - Tapping on the "+" button will allow you to add a new train. Tap a field and...
6 - ... the keyboard pops up. Tapping cancel will cancel input, tapping save will save a new entry in the database.
7 - Tapping on the trash can button allows you to delete trains. Tap on the red circle button and...
8 - ... you'll get a confirmation to delete the train. Tap the "delete" button and the train is gone. Tapping "done" will leave delete mode.

9 - Searching is also possible
10 - It'll search on both train names...
11 - ... as well as train descriptions.

Clicking on the blue circular button should bring you to a detail view, but I haven't started on that yet.

[Martijn Meerts]

Quick update.. Haven't been able to do much due to work...


I've managed to make a WiFi connection between an iPhone app and an OS X app, which means I can send data from the iPhone app to the OS X app, which can then get translate into something that the digital interface understands.

So, for example, in the iPhone app I'm controlling a locomotive with DCC address 3. I can move a speed slider which fires some code in the iPhone app which reads the value. I then create a bit of data that includes the locomotive address and the new speed step. That data gets sent to the OS X application over WiFi, where it'll be translated into the hex values that the Lenz interface needs. And, if all goes well, the locomotive will change its speed =)


In short, I have all the core functionality functioning now, I "just" need to build the necessary components around it =)

[cteno4]

Martijn,

Hey congrats, i know that feeling when you complete that control chain from interface to device, its a great feeling as then you know you can do what you need to! I remember trying to figure out a control table/matrix in hex for a sony projector that had everything else written in japanese (tech docs had not yet been translated on the new model). lots of poking and fiddling, but that wonderful feeling when you get the rosetta stone and you push the button and the action happens!

cheers

jeff

[Martijn Meerts]

Yeah.. iPhone development actually makes that easy on you.. Not specifically the part where you get the core to work, but writing something for the iPhone, and then seeing it work on the actual device itself rather than a simulator is really stimulating.. That definately helped me with getting as far as I have so far.

Also, the Objective C syntax is starting to become familiar, and I can understand sample code much better by now, so it's all positive so far =)

[cteno4]

Im envious! would love to work on some iphone apps, but C is just way, way too much pain with my dyslexia. just takes all the fun out of it and puts the frustration way too high...

keep playing, love to see where this all leads!

cheers,

jeff

[quinntopia]

Wow! That looks amazing!  To be honest, I've always been a bit skeptical of touch screen devices (still am), and wasn't at all sure that the form factor of an iPhone would be something that would feel 'right' to run trains with (hmmmm....I'm starting to sound like an anti-DCC luddite now! Remember those DC only folks who like the big throttles?  ), but after seeing your screen shots...that could be really cool! I'm convinced!     It makes all the other handheld controls look so antiquated. Congratulations!

[Martijn Meerts]

I was quite skeptical about the typing on the iPhone, but it's actually nowhere near as bad as I feared.

The single most biggest advantage of a touch screen is that you can adjust the interface to what you're doing. Picking a train can use the entire screen (compared to a small screen showing only 2 or 3 lines), so it's easy to pick a train. Once you've picked one, the entire surface becomes the controller. Heck, you can even set it up so that people can choose if they want to use buttons, a slider or a virtual knob for speed control =)

[CaptOblivious]

I need a physical keyboard to type. That's why I have a t-mobile G1 (will exchange for a Moto Cliq soon). All the advantages of a touch screen with a keyboard :)

[Martijn Meerts]

That's what I thought.. Now I'm used to my iPod Touch keyboard, and hate typing on my mobile phone ;) (I don't have an iPhone, yet ;))

[Martijn Meerts]

Little not-so-spectacular update, but a big deal for me ;)

I'm using a database to store trains, and before, whenever I changed the structure of the database (say, adding a field for installed decoder), all data was lost because the old database wasn't compatible with the new one.

So, had some time recently to look at the problem, and it seems functionality to migrate old data to a new database structure is actually real easy, and I've managed to implement it =)

The disadvantage is that now I have nothing left to research, and I have to start designing an interface for both the OS X app that connects to the digital central control, and the throttle view on the iPhone/iPod Touch... So, the part I dread the most =)

[cteno4]

Martijn,

they that progress! what about potential of using benito on the osx side for db? pretty flexible system and seems to want to hook into everything else. might save a lot of interface work and allow more user flexibility that you would not have to build yourself. very inexpensive product.

cheers

jeff

[CaptOblivious]

Bento doesn't strike me as the best option for hooking into from a third-party app. First, it would require anyone who used Martijn's program to also have a copy of Bento (which, as I recall, isn't free). Second, OSX has some pretty good internal APIs for this sort of thing anyway (I'm thinking here of Core Data)...

Martijn,

they that progress! what about potential of using benito on the osx side for db? pretty flexible system and seems to want to hook into everything else. might save a lot of interface work and allow more user flexibility that you would not have to build yourself. very inexpensive product.

cheers

jeff

[cteno4]

yes it would require bento, but its like $45, not a bad price for all it can do. its made with a lot of the main osx apis to allow hooking into other apps from what i have seen. there are lots of good apis there to roll your own, was just thinking that for just data base front end to not have to roll his own interface and features. also dont have to do a lot of work to make it highly user modifiable. he was talking like this was the part he was not as thrilled at doing... just a thought.

cheers

jeff

[Martijn Meerts]

I hate Bento actually, don't like the UI ;)

Either way, I'd have to write an add-on for Bento that would allow connecting to the digital interface, and I'd still end up having to design a throttle UI, and make a default UI in Bento.

It's not that I hate doing interface code, it's just that I suck at designing interfaces that's the problem. I do have some ideas though, so I'll probably end up making a fairly simple test based on those ideas and work from there.

And yes, I'm using Core Data, great technology that one =)

[cteno4]

i just like it as its a flexible tool and ui is better than filemaker was! actually am planning on using it for my train db for flexibility.

i hear you on the ui stuff. so many permutations/combinations to deal with!

cheers,

jeff

[Martijn Meerts]

I had a look at the somewhat improved UI in Bento 3, it's a little bit better nowadays it seems ;)

Right before I fried the graphics card in my Mac Pro I started working on some UI ideas. I'm leaning mostly towards something as simple as the finder, with a source list on the left, and just a table on the right. And of course a search box somewhere near the top. I think for version 1 of the OSX app that's all I need, and there's no reason to make it any more complicated. Getting core data to work correctly with a source list view will be interesting enough for a start =)

Of course, there won't be any big updates until after I've moved and settled in, which may take a while.

[Shashinka]

That's what I thought.. Now I'm used to my iPod Touch keyboard, and hate typing on my mobile phone ;) (I don't have an iPhone, yet ;))

I'm actually replacing my netbook with the iPod Touch/iPhone because I found the keyboard to be much easier to work with than the blackberry quarty keyboard. Can't comment much about bento or api for osx. I need to learn sooner or later becasue I'm thinking of making a few apps myself that are user specific.

[cteno4]

martijn,

fyi here is an opensource iphone throttle app starting.

cheers

jeff

 http://github.com/toholio/Open-Touch-Throttle

[Martijn Meerts]

martijn,

fyi here is an opensource iphone throttle app starting.

cheers

jeff

 http://github.com/toholio/Open-Touch-Throttle

Could be interesting to look at how they do it. Right now the iPhone app I have is pretty similar to this, except that I don't hook it up to JMRI, but some custom test program for now ;)

[quinntopia]

Related to this topic...Alex at DCCtrain demos an Esu Ecos command station using an Ipod touch as a Wifi throttle.  The future is here.  These are wonderful times for this hobby.
<a href="http://www.youtube.com/v/HCVHfEIFf7s" target="_blank">http://www.youtube.com/v/HCVHfEIFf7s</a>

[Martijn Meerts]

The program they're using is TouchCab, which works with the ECoS (I and II), Marklin Central Station (1 and 2 I believe), and recently also with the new Lenz Interface which also has an ethernet port.

I found the program rather limited though. I've only tried the free version to get a feel of the program, but I believe it could be done easier. Whoever made it decided to not use the Apple interface guidelines for designing the thing.

[quinntopia]

I found the program rather limited though. I've only tried the free version to get a feel of the program, but I believe it could be done easier. Whoever made it decided to not use the Apple interface guidelines for designing the thing.

Quite true...I've been watching several videos of users on YouTube demo'ing this, they all seem to fumble with the speed quite a bit. 

I have to say...I become more and more impressed with Ecos all the time, although its definitely on the 'high end' of command stations, it seems far more of a value than the Central Station from what I've seen.

[Martijn Meerts]

ECoS and Central Station are about the same. ESU made the first Central Station for Marklin (and ESU still supports the first Central Station), the seconds Central Station is based very much on the first one.

Central Station 2 is faster and design wise (both hardware and software) much nicer than the ECoS, but the ECoS has better support. There have been complaints about the ECoS being too slow however.

[inobu]

I found the program rather limited though. I've only tried the free version to get a feel of the program, but I believe it could be done easier. Whoever made it decided to not use the Apple interface guidelines for designing the thing.

Quite true...I've been watching several videos of users on YouTube demo'ing this, they all seem to fumble with the speed quite a bit........

It looks like they (I-app designers) designed the throttles off of written design note only. They should also take physical operating characteristics of the throttle into consideration. Most throttles are operated with two hands and knob controlled. Trying to get precision response from thumb controls is difficult. We use the thumb for course adjustments or on/off control and the index finger is used for fine adjustments.

If they are going to use the thumb for fine adjustments then the buttons should be configures as such. Lenz

More details...

Funny how one can see the differences in design guidelines.


The I-apps design guidelines need to eliminate the issues of the other units to become successful.

Inobu

[cteno4]

yes a thumb control would need to be vertical, not horizontal for both operational direction and getting more space to give wider area of movement. idea of the nce thumb wheel is also good as it has no 0 point, just keep spinning to increase/decrease rate. very handy when switching between trains as knob is then at current speed of train you are switched to and go from there up or down. you get the analog advantage (over digital up / down buttons) of the wheel, but freedom of multiple spins for finer control if you want.

cheers

jeff

[Martijn Meerts]

The problem is personal preference. There are plenty people who like TouchCab and happily use it. Personally, I think it looks terrible =)

Looking at the physical throttles, they're all ugly except the one from Lenz. I have the Lenz one and while it functions well enough, I prefer speed control using a knob. The Lenz controller with knob is terrible though ;)

This is where a touchscreen is nice. You can code an app that allows user to pick a throttle layout. A slider for speed (like TouchCab), or a knob that goes from 0 to 28 speed steps, or 0 to 127 speed steps, or a knob that goes from 0 to 200 km/h, or a continuous knob that has no start or end, or a type of dynamic brake system using 2 levers... It's all possible.

[inobu]

That's my point in that the physical operation and software operation can address most of the fault aspects in the throttle. I think the I-app is the solution in general because of the ease in which the interface can be changed or redesigned.

Skinz and programmable function strings can address personal preference, its just the type or types of interfaces (button, locations etc..) that needs to be address. The thumb plays a vital role in I-apps because of the on-hand rule (so to speak) but for precision control the thumb need either touch sensitive control or touch duration control in the right location. The thumb has limited reach and response in most cases and I-apps are very dependent on it.

There are many aspects to design and it is not only operational but ergonomics plays a vital role. It seems most companies only want to choose one or the other.

Inobu 

[Martijn Meerts]

The thing to be careful with though, is that if you give users too much customizability, they'll start thinking it's too complicated. In the case of a throttle, you can supply several pre-defined throttle layouts, but I don't think they should be allowed to customize button locations and such. If you have a knob control (or iPad classic click wheel like control), you could allow users to set sensitivity though, or base the sensitivity on the train type (shunting loco needs more sensitive control than a high speed train for example).

Right now I'm playing a little with a knob control on the iPhone, to see how it works, and if it's usable. My math isn't the best though, so I'm in an epic battle with radians, atan, angle calculations and an axis system that I'm not entirely used to =)


What we could do, is try to come up with some ideas of how we would like a throttle (specifically an iPhone/iTouch throttle) to look, what kind of features we'd want etc. From there I could try to code some mockups to see if they work on the actual device. Obviously I'd be happy to share code with anyone who wants it =)

[inobu]

I think the buttons have to be based off thumb control primary and index secondary.

More details...

The blue button to right are speed control. coarse control and fine control. The green should be some type of indicator of slider.
Center is direction.

026 speed read out
5001 unit number

Rough draft

Inobu

[cteno4]

inobu,

nice!

yes this is what i was thinking of the thumb oriented layout that is more vertical than horizontal for thumb control.

slider/indicator is nice as you could set a speed to accelerate or brake to via the slow or fast buttons or just slide it. should be able to have accelerate and brake features as well with the dcc. can these control systems for the master do acceleration and brake rates on the fly or only use the rate set on the decoder program? sorry i m not up on all the dcc features of all the different systems. tough to get it all in in small screens, start seeing the reasons why the dedicated controllers start getting big!

needs a loco menu button or some sort of toggle for the loco list/selection.