A different idea relating to scripting (not LUA)

openTx has introduced a range of new features, ideas and bling. It is fast becoming the firmware of choice for many users. openTx will run on ALL current hardware platforms, including the gruvin9x and sky9x boards. Work has already started to support the new FrSky X9D radio!
nischris
Posts: 48
Joined: Sat Feb 08, 2014 10:39 pm
Country: -

A different idea relating to scripting (not LUA)

Post by nischris »

Hi there

Firstly, please forgive the rather long-winded post.

I have seen the recent work put into the support for LUA on OpenTX and have also read some of the concerns that people have with LUA e.g.
Increased CPU load
Amount of memory required
Increased firmware size
Learning cure associated with LUA
Infinite loops in the scripts
Memory leaks
Script size

There are probably other concerns people have too, but those concerns seemed valid. I realise that these are _concerns_ and not necessarily problems that cannot be overcome, and I'm sure that many of them probably already have been.

Anyway, these concerns got me to thinking about a system that I used in a previous life when I worked as an air conditioning control systems engineer. We used a system that was programmed using what they (Alerton) called "DDC", or "Direct Digital Control". This 'language' comprised of logical objects joined together using input/output items. Examples of typical objects would be:
logical OR
logical AND
logical XOR
logical NOT
Comparator (compare one analog input to another)
Switch (output switches between two analog input values depending on digital control input state)
PID loop Add (and friends)
Multiply (and friends)
Delay on make timers (output turns on once the input has been on for a period of time)
Delay on break timers (output turns off once the input has been off for a period of time)

The input/output (IO) items represented things like raw analog input values, switch states, internal storage (used to contain output values of objects), relay outputs, analog outputs, and values to be presented to the user, settings provided by the user, and constant values.
The logical objects could pretty much use any IO items as inputs, and also output to any IO item (where that makes sense). This made the system very flexible, as each IO item was treated exactly the same way by every object. e.g. a logical AND object would treat a physical switch input in exactly the same way as the output from some other logical output, or (say) a physical output state.
This system had quite a few quite neat features:
e.g.
The 'programs' tended to be quite compact. A 'DDC' file to control a chiller/boiler/pump set/air handler would typically be in the order of a few KB long.
Minimal setup time in the firmware (basically involved reading the 'DDC' file into memory).
Fast execution time
Relatively fixed execution time (within reason) for each pass of the program
Relatively small code size required to support the DDC
Relatively small memory requirements

When running the program, the device would process each logical object in turn until the last object was processed. The device would repeat this process at regular intervals (e.g. 100ms, but any rate is fine as long as it doesn't exceed the time taken to execute the complete set of objects). Each object would read its associated input values, perform whatever operation it was designed to do (AND/OR/Switch etc), and write its output value (or values), if any. Some objects would need to retain some state information between passes (e.g. one-shot), and this would be retained within the object memory taken up by that object.

I found this system of programming entirely intuitive, although it has to be said that some people found it very difficult to understand, so what do I know.
There were some drawbacks to using this system as well...
e.g.
The programming interface was pretty clunky. It was difficult to get a printout of the 'DDC' in a visual form that was easy to follow. Typically, a simple textual listing was available, but that was about it. It could be difficult to follow if you weren't used to it. Generally, the engineer would wind up drawing the 'program' out by hand on a piece of paper from the listing. Of course, the engineer could draw out the program on the PC using some kind of package like AutoCAD, Visio or whatever, but there was generally no linkage between the program file and the drawing, so wheneever the program changed, the drawing would need to be manually updated.

Anyway, it seemed to me that this style of program might fit the Taranis transmitter quite nicely. The transmitter has a number of analog and digital inputs that need to be manipulated/altered/mixed in some way, and various operations need to be done with the results (beeping/playing MP3,LCD popups etc).

So, I thought I would see if I could come up with something that would serve as a proof of concept and give other people a chance to see how it works, and provide some feedback about what they thought of it.

I set about creating some code that would add this sort of support into the OpenTX firmware. I have created a simple editor program that allows the user to generate a 'DDC' file. The set of objects I have supported is by no means complete, but does give a bit of an idea of what is possible.
Currently supported objects:
2 input OR
4 input OR
2 input AND
4 input AND
2 input XOR
NOT (output is OFF if input is ON)
Comparator (compares two analog values. Output is ON if plus is > minus. Output is OFF if plus is <= minus - deadband
One shot. Output is ON for one scan when the input first turn ON, else the output is OFF
Beep. The device will cause the Taranis to emit a 'beep' of the specified frequency when the input first turns ON.
End. Does nothing other than indicate that this is the last object to be processed in the scan. Not generally required.

Currently supported IO types
Constant. Constant integer values like 0, 100, -10.
Branch. Memory used by the DDC code as temporary storage. Objects can read/write to branches as they wish. The branch values are retained from one pass to another. Can be used as counters, timers, that sort of thing.
Calstik. Calibrated stick values from the Taranis. These IO values are taken directly from the 'calibratedStick' table in the OpenTX firmware. They have a range from -1024 to 1024. calstik 4 appears to map to the pot on the LHS of the Taranis, and calstik 5 appears to map to the pot on the RHS of the Taranis.
LAI. Not fully implemented yet, but the intention is that these values would be made available for the Taranis to use as inputs to things like mixes or custom functions. They would provide a way for the DDC code to supply analog values back to the Taranis.

Note that IO values could basically be taken from wherever we want. e.g. Raw ADC values, physical switches (inc. menu/trim), mixer outputs, channel outputs, telemetry values. The DDC firmware just needs support for that that IO type adding into it and the the DDC program can start using it. The way I have written the code, it is a relatively easy task to add new IO types/objects. I have found that the tricky bit is dealing with the C <-> C++ namespaces.

Also, there is pretty much no limit to the type of DDC objects that can be supported.
e.g.
PID loops
scaling
timers
rate limiting
maths operations (e.g Sin/Cos/Tan, square root)
random number output
value limiting
Taranis specific stuff
e.g.
beeping
playing MP3
drawing to the LCD
setting trim values
exponential curves
raw stick
value->percent

As well as creating the DDC editor program, I have modified the OpenTX firmware so that it will execute the DDC. In addition I have created a sample DDC file to serve as an example of what can be done with this system. The file contains 14 object blocks and causes the Taranis to 'beep' at 5 different frequencies depending on how much higher the LHS pot is above the RHS pot.
I have attached some OpenTX firmware (based upon release R2940) that includes the support for running a DDC file. At present, the DDC file must be called "1.ddc" and must be installed into directory "/ddc" on the SD card.

I have included the executable (Linux version only. The Windows version requires Cygwin DLL) that allows the user to create new DDC files. Note that this executable is _very_ basic. It allows new objects to be created, delete objects, load existing files (via the command line) and save the file.
Very quick tutorial on how to edit DDC:
*) Draw out the DDC diagram on a piece of paper first. This helps to visualise how the program will work.
*) Each DDC object has a seqence number. The device executes each object in order, from lowest to highest sequence number. Use the '+' and '-' keys on the keypad to move between objects. The END key will take you to the last object, and the HOME key will take you to the first object. Leave a space between each object number (default is 10) to allow room for new objects to be inserted.
*) Once at the desired sequence number, select the object type to insert. (e.g. AND/ADD/Comaparator. Some new text should be displayed below the object type once an object has been selected.
*) Move to each IO field in turn and assign the desired IO type and instance into each field.
*) repeat until done.

Note that the size of the DDC file is determined by the number of DDC objects your program has in it. The IO types and values make no difference to the file size.

example commands for the editor program:
./dedit -f 1.ddc -o edit Edit file 1.ddc
./dedit -f 1.ddc -o print Print file 1.ddc
./dedit -f 1.ddc -o renum -s 10 Renumber 1.ddc with a sequence spacing of 10

Renuumbering the file is useful if a number of new objects have been squeezed betwen

Note that there a number of ".txt" files included in the attached .zip file. They are referenced by the executable file and need to be installed into the same directory as the executable.
Once you have created your DDC file (or just wish to try my example), transfer it into /ddc/1.ddc on the SD card of the Taranis. Install the supplied firmware onto the Taranis and you should be good to go. Again, this firmware is based up r2940, so be sure to save your eeprom file before experimenting with this firmware. The DDC file is loaded when a model is loaded, and is currently executed every 10 ms (just after the mixes have been processed).

I have also attached a very rough image (beeper.pdf) representing what the attached ddc file does.

I'd be very interested to hear what people think of this system.

If there is sufficient interest, I might look further into adding more complete support into OpenTX. At the very least, the firmware needs to support running different DDC files depending on the selected model, more object types and more options relating to IO types. If this code is something that the community would like to incorporate into OpenTX, I would definitely look at donating it to the cause.

If anybody would like to try this, but can't get it to work after following the instructions, let me know and I'll see if I can help out.

Cheers
Attachments
ddc.tar.gz
contains modified r2940 OpenTX (for the Taranis) firmware. DDC file editor (dedit), example ddc program (1.ddc) and associated .txt files required by the DDC editor. Install the firmware into your Taranis (save your old firmware and EEPROM files first!), copy the file 1.ddc into /ddc directory on the SD card, and you should be good to go.
(207.5 KiB) Downloaded 305 times

bertrand35
9x Developer
Posts: 2764
Joined: Fri Dec 30, 2011 11:11 pm
Country: -

Re: A different idea relating to scripting (not LUA)

Post by bertrand35 »

I am not sure it will be easy to write custom templates with DDC files?
nischris
Posts: 48
Joined: Sat Feb 08, 2014 10:39 pm
Country: -

Re: A different idea relating to scripting (not LUA)

Post by nischris »

I hadn't considered that the DDC would be used for this application, but no, I don't think it would suit that purpose very well. I'm not really sure how custom templates work btw, but I assume there is a fair amount of writing to the LCD and interaction with the user.
It is more aligned with control of the device rather than GUI type applications. I think I have an inkling of the power that LUA provides, and quite like what it can do. I really wanted to see if I could get this 'DDC' working on the device because it seemed easy, small (code and memory requirements), fast (relatively).
bertrand35
9x Developer
Posts: 2764
Joined: Fri Dec 30, 2011 11:11 pm
Country: -

Re: A different idea relating to scripting (not LUA)

Post by bertrand35 »

Why not both?
nischris
Posts: 48
Joined: Sat Feb 08, 2014 10:39 pm
Country: -

Re: A different idea relating to scripting (not LUA)

Post by nischris »

No reason at all I think. I realise that this DDC isn't as flexible as LUA, and some people would definitely prefer to use it.
I guess that total firmware size/memory requirements might be an issue, although DDC requirements are fairly small.

User avatar
jhsa
Posts: 19480
Joined: Tue Dec 27, 2011 5:13 pm
Country: Germany

Re: A different idea relating to scripting (not LUA)

Post by jhsa »

I don't understand much about programming but here is a question..
Would that run on a sky board?

Sent from my GT-I9195 using Tapatalk
My er9x/Ersky9x/eepskye Video Tutorials
https://www.youtube.com/playlist?list=PL5uJhoD7sAKidZmkhMpYpp_qcuIqJXhb9

Donate to Er9x/Ersky9x:
https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=YHX43JR3J7XGW
nischris
Posts: 48
Joined: Sat Feb 08, 2014 10:39 pm
Country: -

Re: A different idea relating to scripting (not LUA)

Post by nischris »

I don't know anything about that board, but I suppose that as long as there is enough flash free and enough memory available, and SD card support, it probably would work ok.
If they're considering putting LUA support on that board then I'd say that it would definitely run on it. If there aren't enough free resources on the Sky board for LUA, there might still be enough for this DDC code.
I have only worked on getting it working on Taranis because that is the type of TX I have.
nischris
Posts: 48
Joined: Sat Feb 08, 2014 10:39 pm
Country: -

Re: A different idea relating to scripting (not LUA)

Post by nischris »

Oh, if you're talking about the firmware I attached, then definitely not.
User avatar
MikeB
9x Developer
Posts: 17990
Joined: Tue Dec 27, 2011 1:24 pm
Country: -
Location: Poole, Dorset, UK

Re: A different idea relating to scripting (not LUA)

Post by MikeB »

I don't know linux much. I have a virtualbox running linux and tried the first of your example commands from a terminal window and just get bash reporting it cannot execute a binary file.
For those who don't know linux (I'm running lubuntu), can you give more instructions on how to run dedit?

Specifically, you have got this working in openTx on the Taranis. Ersky9x is also available to run on the Taranis. Both openTx and ersky9x also run on the SKY board (Atmel ARM processor).

Mike.
erskyTx/er9x developer
The difficult we do immediately,
The impossible takes a little longer!
nischris
Posts: 48
Joined: Sat Feb 08, 2014 10:39 pm
Country: -

Re: A different idea relating to scripting (not LUA)

Post by nischris »

I wouldn't say I'm any kind of linux expert either.
Just rechecked what I wrote as an example command. The last few words are not part of the command, so it should be
./dedit -f 1.ddc -o edit
Perhaps that's the problem, although it seems like your terminal doesnt want to run the executable at all. Perhaps the file ownership needs to be changed, or the file mode (using chmod command) needs to be set.
The output from an "ls -al" command might be handy to see if it won't run. Even just entering the command ./dedit should produce something from the program
User avatar
MikeB
9x Developer
Posts: 17990
Joined: Tue Dec 27, 2011 1:24 pm
Country: -
Location: Poole, Dorset, UK

Re: A different idea relating to scripting (not LUA)

Post by MikeB »

A directory listing shows it as an executable. Double clicking says it is an executable, and if I select execute, nothing shows.
It is only 44.3K, is this the complete program, or does it need installing in some way with other support files?
In a terminal box (tried two different ones), entering ./dedit just gets bash reporting "cannot execute binary file".

Mike.
erskyTx/er9x developer
The difficult we do immediately,
The impossible takes a little longer!
bertrand35
9x Developer
Posts: 2764
Joined: Fri Dec 30, 2011 11:11 pm
Country: -

Re: A different idea relating to scripting (not LUA)

Post by bertrand35 »

what does say 'file <this bin>' ?
User avatar
MikeB
9x Developer
Posts: 17990
Joined: Tue Dec 27, 2011 1:24 pm
Country: -
Location: Poole, Dorset, UK

Re: A different idea relating to scripting (not LUA)

Post by MikeB »

ELF 64-bit LSB executable x86-64 . . .
That explains it, I'm running a 32-bit version of linux.

Mike.
erskyTx/er9x developer
The difficult we do immediately,
The impossible takes a little longer!
nischris
Posts: 48
Joined: Sat Feb 08, 2014 10:39 pm
Country: -

Re: A different idea relating to scripting (not LUA)

Post by nischris »

Yes, I just woke up and it came to me just now. I am running 64bit windows/linux. I wondet if I can build 32bit versions.
Yes, the whole program is pretty small.
User avatar
MikeB
9x Developer
Posts: 17990
Joined: Tue Dec 27, 2011 1:24 pm
Country: -
Location: Poole, Dorset, UK

Re: A different idea relating to scripting (not LUA)

Post by MikeB »

My windows is 64-bit (win-7), but I made the virtualbox 32-bit.

Mike.
erskyTx/er9x developer
The difficult we do immediately,
The impossible takes a little longer!
User avatar
Kilrah
Posts: 11108
Joined: Sat Feb 18, 2012 6:56 pm
Country: Switzerland

Re: A different idea relating to scripting (not LUA)

Post by Kilrah »

nischris wrote:If they're considering putting LUA support on that board then I'd say that it would definitely run on it. If there aren't enough free resources on the Sky board for LUA, there might still be enough for this DDC code.
Sky9x doesn't have enough flash for LUA.
The 9XR-Pro might have, as if I remember well it has double the amount.
User avatar
MikeB
9x Developer
Posts: 17990
Joined: Tue Dec 27, 2011 1:24 pm
Country: -
Location: Poole, Dorset, UK

Re: A different idea relating to scripting (not LUA)

Post by MikeB »

I got HK to fit the '3S8 (512K flash) so the possibility was there.
Why does LUA take up so much flash anyway?

Mike.
erskyTx/er9x developer
The difficult we do immediately,
The impossible takes a little longer!
User avatar
Kilrah
Posts: 11108
Joined: Sat Feb 18, 2012 6:56 pm
Country: Switzerland

Re: A different idea relating to scripting (not LUA)

Post by Kilrah »

It's quite a big bunch of code... takes about 120kB.
On sky9x opentx already takes 177kB. On Taranis (without lua) about 277kB.
nischris
Posts: 48
Joined: Sat Feb 08, 2014 10:39 pm
Country: -

Re: A different idea relating to scripting (not LUA)

Post by nischris »

OK, here's an archive that contains a Windows (64 bit :( ) version of dedit.exe. Also included is a new 1.ddc file that is slightly different from the first example. This one makes use of a timer. New opentx.bin file also included.
Remember that this code is _very_ new and the DDC objects may not all work pefectly (not expecting too much to be wrong).
I do realise that the DDC support currently doesn't do anything that you can't already achieve with custom functions/switches, but I think it has the scope to grow beyond what they can do.
Attachments
ddc-2.tar.gz
Includes new opentx.bin, windows 64bit version of dedit.exe (and cygwin1.dll) and slightly different 1.ddc example file.
(1.26 MiB) Downloaded 293 times
User avatar
MikeB
9x Developer
Posts: 17990
Joined: Tue Dec 27, 2011 1:24 pm
Country: -
Location: Poole, Dorset, UK

Re: A different idea relating to scripting (not LUA)

Post by MikeB »

It just seems a bit large, is it written by people who haven't come from a background where flash and RAM are limited?
Having come from using "tiny Basic" (integer only) that fitted into 2K of a Z80, and a real Basic interpreter that ran in 5K (full floating point), and these effectively included an editor as well as the runtime interpreter I just question why this code needs to use 120K?

Mike.
erskyTx/er9x developer
The difficult we do immediately,
The impossible takes a little longer!
User avatar
MikeB
9x Developer
Posts: 17990
Joined: Tue Dec 27, 2011 1:24 pm
Country: -
Location: Poole, Dorset, UK

Re: A different idea relating to scripting (not LUA)

Post by MikeB »

nischris wrote:OK, here's an archive that contains a Windows (64 bit :( ) version of dedit.exe.
Running it on windows and it can't find:
cygformw-10.dll
There may be other dlls missing, that is the first it fails on.

Mike.
erskyTx/er9x developer
The difficult we do immediately,
The impossible takes a little longer!
User avatar
Kilrah
Posts: 11108
Joined: Sat Feb 18, 2012 6:56 pm
Country: Switzerland

Re: A different idea relating to scripting (not LUA)

Post by Kilrah »

Good question, I guess the best way is to look at it... but today places where I've seen it used is usually machines with dozens or hundreds of MBs of memory (smartphones, set-top boxes etc).
nischris
Posts: 48
Joined: Sat Feb 08, 2014 10:39 pm
Country: -

Re: A different idea relating to scripting (not LUA)

Post by nischris »

MikeB wrote:
nischris wrote:OK, here's an archive that contains a Windows (64 bit :( ) version of dedit.exe.
Running it on windows and it can't find:
cygformw-10.dll
There may be other dlls missing, that is the first it fails on.

Mike.
Hmm, as you say, there may be other dependencies. Not sure how to tell what's required given that my machine here obviously has everything it needs already set up. As the exe uses ncurses, as that missing dll looks suspiciously like it's related to one of the ncurses libs, I'll post them all up.
(If I'd known anything at all about writing windows apps, I would have gone that way, but because I don't, and I wanted to get something working asap, all I have is this console app with cygwin dependencies. :( ).
Attachments
cygwin-dlls.tar.gz
(194.75 KiB) Downloaded 211 times
User avatar
Kilrah
Posts: 11108
Joined: Sat Feb 18, 2012 6:56 pm
Country: Switzerland

Re: A different idea relating to scripting (not LUA)

Post by Kilrah »

C:\Users\Andre\Desktop\ddc-2.tar>dedit -f 1.ddc -o edit
operation: edit
Error opening terminal: cygwin.
A bit better. Guess it's needed to properly install cygwin. Won't do it as last time I tried that sort of stuff it broke my opentx build system and I spent a day getting it to work again.
nischris
Posts: 48
Joined: Sat Feb 08, 2014 10:39 pm
Country: -

Re: A different idea relating to scripting (not LUA)

Post by nischris »

Hmm. Sorry about that. You can build opentx under windows? Easy to set up?
User avatar
Kilrah
Posts: 11108
Joined: Sat Feb 18, 2012 6:56 pm
Country: Switzerland

Re: A different idea relating to scripting (not LUA)

Post by Kilrah »

Yeah I do everything on Windows. Installing Yagarto, WinAVR and Python is all that's needed for the firmware if I remember well.
nischris
Posts: 48
Joined: Sat Feb 08, 2014 10:39 pm
Country: -

Re: A different idea relating to scripting (not LUA)

Post by nischris »

but not Cygwin I guess. I might have a look at setting up a 32bit Linux VM and seeing if I can build everything in that. Big sigh. I only went the Cygwin way because I couldn't figure out how to build with the ncurses support under mingw. I might revisit that too.
nischris
Posts: 48
Joined: Sat Feb 08, 2014 10:39 pm
Country: -

Re: A different idea relating to scripting (not LUA)

Post by nischris »

OK, here's a 32bit Linux version of dedit.
Cmd to edit a DDC file (called 1.ddc): ./dedit -f 1.ddc -o edit

It may be that you need to have ncurses installed on you machine (a "file" command indicates that dedit uses shared libraries). If you need to have ncurses installed, the command is...
sudo apt-get install libncurses5-dev libncurses5
Attachments
dedit.tar.gz
32bit Linux version of dedit.
(15.65 KiB) Downloaded 157 times
nischris
Posts: 48
Joined: Sat Feb 08, 2014 10:39 pm
Country: -

Re: A different idea relating to scripting (not LUA)

Post by nischris »

In case anybody is interested, I've been plugging away with my little project. I've been adding support for various objects and IO types.
Some new objects include:
Get mixer settings (add/set mixer settings coming soon)
Play a file
Play a value (obtained from getValue() function). Includes units if desired.
Get value from 'anas[]' and 'rawAnas[]' tables
Flipflop (bit like a regular SR latch)
Add, subtract (multiply, divide coming soon)
New IO type:
Switch value (from getSwitch() function). Means that DDC can see pretty much all physical/logical switches.

I haven't put any effort into writing to the LCD at this stage. Although I'm sure it's possible, I think LUA might be better suited to that task.

Out of interest, I created a wee script that performed the countdown request that I've recently seen queries about. I was a bit rusty at writing DDC scripts, but it only took a few minutes to create, used about 15 objects, and works pretty much as the user was requesting. i.e. audio at 30 seconds, 20 seconds, 15, and 10-0 seconds. Enabled using SWA. This script created a file ~900 bytes, which is also all the memory that is required when running it.
I haven't made any effort to try an keep the file size to a minimum at this stage, but I'm sure that if space was a real issue (don't think it is when compared to LUA), the memory requirements could be reduced significantly.
User avatar
jhsa
Posts: 19480
Joined: Tue Dec 27, 2011 5:13 pm
Country: Germany

Re: A different idea relating to scripting (not LUA)

Post by jhsa »

This is getting interesting as far as the skyboard is concerned.. :)
Lets see what Mike says after trying it ;)

João

Sent from my GT-I9195 using Tapatalk
My er9x/Ersky9x/eepskye Video Tutorials
https://www.youtube.com/playlist?list=PL5uJhoD7sAKidZmkhMpYpp_qcuIqJXhb9

Donate to Er9x/Ersky9x:
https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=YHX43JR3J7XGW

Post Reply

Return to “openTx”