Advancement of the telemetry protocoll

[rainer]

Todays implementation of the frsky telemtry protocoll is limited to the data being transmitted of existing frsky sensors. With projects like the openXvario/openXsensor and hopefully more in the near future, we are evolving into a situation where the protocoll does not meet our requirements any more.
As an example I put together a spreadsheet containing a list of all the available telemetry data in the upcoming openXsensor. This list is meant as a help for any discussions regarding new IDs for the 9X firmwares, the FRsky dashboard or any other system that wants to make use of the available data.
The current way the FRSky telemetry protocol is implemented, we are missing a good solution to transfer and use that data on the TX/ground station.
= openXsensor - list of the available telemetry data. =


https://openxvario.googlecode.com/svn/w ... 20Data.ods

As of today some of the data can only be transmitted by res-/mis-using other fields that are officially or unofficially (like the DIST field) writable via the telemetry protocol.
But as all of the available solutions are open source, why not start a discussion about implementing a new way of transfering data?

== Idea 1 - create new IDs with predefined names, IDs, units,... ==
One idea would be for example to agree on new dedicated IDs fields as we need it. These fields could be used like the existing telemetry fields with predefined field names and units.

== Idea 2 - create new configurable "Generic Data Fields" ==
Another way would be to create new "generic data fields".
similar to the way this is currently implemented in the FRsky dashboard app where you define the field, choose the unit,precision,name and so on to create fields as we need them.

== Idea 3 - Plug and Play Sensors. ==
let the sensor introduce itself to the tx/ground station.
Why not give the sensor the possibility to send some data about itself to the tx/groundstation?
When the sensor powers up it would send some packages about some new ID's describing its basic properties.
* Name of the sensor as e.g. 4Bytes (spread over various packages if needed)
* Unit used from an enumeration of available units
* Datatype from an enumeration of available datatypes
* Number of decimals
* ....

I know that at least for the 9x firmwares we are always low on resources, but with an open source protocoll alternative this would make the existing telemetry implementation (or at least parts of it) obsolete.
For other systems like the taranis, the sky9x board or the atmega128/2561 modded boards, this could bring telemetry to another level.
For smartphone apps interpreting the telemetry protocoll, this should be possible as there is not the resource limitation we have on the 9x.
But in my opinion, the best solution would be one that could reach the most users. Building something that only the taranis could use is only half the fun.

Sorry for the long post... but please treat this post as a request for discussion.

rainer.

[Kilrah]

Problem is the overhead.

IF you want to have your own protocol, but through an frsky receiver then you'll receive that protocol framed in frsky's protocol, ending up with a mix of things. Then the user will have to choose between frsky protocol and "your protocol" depending on what model and sensors he has, when he's always using frsky telemetry gear (confusing).
Then what do you do when you want to combine the openxvario with an frsky hub and gps, or other frsky sensors?

[rainer]

we could use the frsky frame package including the overhead and fill it with new ID's, for the sake of compatibilty...

[rainer]

idea#2 above for example. this would mean e.g. <n> new ID's that would be treated in the tx similar to the A1/A2 today. we use the tx/groundstation to define the unit, number of decimals and the name. then we could have working voice readouts including correct units. and we would only have to put <N> new ID's ( or maybe pair of IDs for higher precision)
the devices not knowing these ID's would simply ignore them, so it should be well possible without compatibility problems.

[Rob Thomson]

we could use the frsky frame package including the overhead and fill it with new ID's, for the sake of compatibilty...

This to me seems to be the best option. Guess someone will need to talk to Eva about it?

Sent from my GT-I9300 using Tapatalk 2

[rainer]

if e.g. ram is too limited to have user definable names for fields, wouldn't it be possible to use companion to add those field names to the Flash before programming? via defines on the compile server or via direct modification of the binary?

[Kilrah]

...except that the frsky protocol is completely changing in the meantime with the switch to the smart port system...

It is however a bit more custom-stuff-friendly already.

[ReSt]

Basic question:
I know that the receiver repackages the data that it reads via its own serial input. As an example, the datastream that the FrSky Sensor Hub sends to the receiver has a different protocol than the protocol the receiver sends via rf to the ground station.

So, does the receiver really repackage every ID that it gets or does it only repackage a certain set of id's he already knows?

Reinhard

[jhsa]

...except that the frsky protocol is completely changing in the meantime with the switch to the smart port system...

It is however a bit more custom-stuff-friendly already.

What about the thousands of people using still the old 2 way receivers? are we getting forgotten?

[rainer]

Basic question:
I know that the receiver repackages the data that it reads via its own serial input. As an example, the datastream that the FrSky Sensor Hub sends to the receiver has a different protocol than the protocol the receiver sends via rf to the ground station.

So, does the receiver really repackage every ID that it gets or does it only repackage a certain set of id's he already knows?

Reinhard

you can use other IDs and the receiver will transmit those packages to the tx modul. I am already doing that for example to fill the GPS DIST field in open tx to have at least one more field where to put some of my data.
The only problem that might occur is a bandwidth limit, but this could be easily handled by using longer transmision intervals where an update every 100ms is not needed.

[MikeB]

Given that FrSky are going down the SPORT route, they are unlikely to add to the existing HUB sensors or to those sensors that send serial data using the HUB protocol without the hub.

There are a number of currently unused IDs in the HUB protocol, 7, 8, 10, 11, 12, 13, 14, 15, 29, 30, 31, 32 as far as I can see.

Some of these could be used as USER fields. The Tx firmware should be able to easily handle these to store them, then it would need some method to define what they mean. The stock Tx may have some trouble with flash space, but all the other boards/transmitters should be able to handle them.

Mike.

[jhsa]

would it be ok on the m128? I'm concerned about the RAM of course..

João

[MikeB]

I saved a significant amount of RAM a while ago so I think we are OK there.

Mike.

[jhsa]

Just thinking here. the case of the openXvario/sensor. the sensor is doing all the calculations so I suppose the tx in this case would only display the data and handle the alarms and the voice. No calculations. Of course would still have to do the calculations for A1 and A2. That part would remain unchanged..
As I normally say, this is my logic and not the real logic


João

[rainer]

Given that FrSky are going down the SPORT route, they are unlikely to add to the existing HUB sensors or to those sensors that send serial data using the HUB protocol without the hub.

There are a number of currently unused IDs in the HUB protocol, 7, 8, 10, 11, 12, 13, 14, 15, 29, 30, 31, 32 as far as I can see.

Some of these could be used as USER fields. The Tx firmware should be able to easily handle these to store them, then it would need some method to define what they mean. The stock Tx may have some trouble with flash space, but all the other boards/transmitters should be able to handle them.

Mike.

Wouldn't it be a better idea to use a continuous range in the upper area? i haven't tested it yet, but am quite sure it would work by using e.g. 0x50 to 0x5F or 0xA0 to 0xAF.
and i still like the idea of using configurable names,units and decimal places. Would this be possible on the TX side?
rainer

[bertrand35]

I think we already tested using the high IDs, but we didn't receive anything on module side! Must be double checked.

On stock board I am also in favor of reusing the IDs. If we don't do that, we will have to implement a new routine to decode and store the telemetry data. It will cost flash ...

Now about USER fields, I am favorable to this idea (perhaps the 6 IDs from 0x0A to 0x0F), provided we find a solution to have the unit and configuration dynamically sent from openXvario. It would avoid to use flash - and bother the user - for the configuration of the custom fields, I am afraid it could be big ... If 15bits are enough for the value (1bit used for the confiuration flag), we could send the ID every 5s with the configuration - or a part of the configuration - (short name, display format, unit).

Bertrand.

[rainer]

I think we already tested using the high IDs, but we didn't receive anything on module side! Must be double checked.

On stock board I am also in favor of reusing the IDs. If we don't do that, we will have to implement a new routine to decode and store the telemetry data. It will cost flash ...

Now about USER fields, I am favorable to this idea (perhaps the 6 IDs from 0x0A to 0x0F), provided we find a solution to have the unit and configuration dynamically sent from openXvario. It would avoid to use flash - and bother the user - for the configuration of the custom fields, I am afraid it could be big ... If 15bits are enough for the value (1bit used for the confiuration flag), we could send the ID every 5s with the configuration - or a part of the configuration - (short name, display format, unit).

Bertrand.

There might be a problem with the oXv sending this data. This would mean that the field will be dynamically "created" from the oXv. how would it be possible to create audible alarms/voice readout if the fields are dynamic? If we don't plug start the sensor, would the fields and their definition still be available for the user to use them in alarms/custom functions?

rainer

[bertrand35]

The voice readout is made of one numeric value and an unit. If the format of the value, plus the unit are sent, it's possible to display them, and also to hear them.

If you don't start the sensor, I think we could just display 0 (without an unit) on the custom telemetry screen, and play nothing on audio.

For the custom switches as well there is not really a problem, we could display the "normal" range of 0-255 (the stored value) when we don't have any info, and the "real" value when we have the multiplier field with the unit.

[Kilrah]

So, does the receiver really repackage every ID that it gets or does it only repackage a certain set of id's he already knows?

When using a D receiver and module AFAIK the "User Data" (=what enters from the serial port) is passed as is with just the framing added.

On the Taranis it is more complicated as the XJT always puts out the data on the smart port, with smart port protocol, regardless of receiver type. So with a D receiver it translates the hub format into the new IDs and representation. Not sure what happens to IDs that are not part of the official hub protocol, or to generic data.

What about the thousands of people using still the old 2 way receivers? are we getting forgotten?

They still work perfectly well. But new sensors will likely be smart port ones, so I wouldn't expect to get new sensors with the old system from frsky.

[rainer]

How about the following idea:
we choose how many new generic data fields we want. some are 16bits big, some are 32 bit.
each will be presented as 1(low precision=16 bits) or 2 (High precision=32 bits) new IDs in the protocol

We then take one additional new ID to use as the setup ID. this ID transmits 2 bytes of data.

• Byte 1 represents the ID we want to setup. (for a high precision field, only the first ID will be used)
• Byte 2 will be split into different values
  - the highest bit is still unused.(might be used to add additional units in the future
  - the next 4 bits are used to define the unit from an enumeration of 16 possible values
  - the 6th bit will be used to define if this should be treated as a signed or unsigned value
  - the lowest 2 bits get used to define the number of decimals to be used when displaying this value.

So in order to define 3 new low precision and 2 new high precision value we would need
3x1 + 2x3 +1 new ID's
and have the possibility to setup unit, number of decimals and signed/unsigned variable type
rainer.

edit: rearanged the bits to put the unused bit in front of the 4 unit bits

[bertrand35]

I am ok with the idea of the setup ID. I don't see where you choose the unit though. Perhaps we don't need the whole "Byte 1" given that we will not have so many custom data fields!

[rainer]

yes i took a wrong word there i'll edit my post above..
it should read:
Byte 2 will be split into different values
- the highest 5 bits are used to define the rangeUNITS from an enumeration of 32 possible values
- the 6th bit will be used to define if this should be treated as a signed or unsigned value
- the lowest 2 bits get used to define the number of decimals to be used when displaying this value.

[rainer]

units could be the following:
00 none no unit specified
01 V Volts
02 mV MilliVolts
03 A Ampere
04 mA Milliamps
05 m/s Meters per second
06 kmh Kilometers per hour
07 m Meters
08 mAh Milliamphours
09 W Watt
10 ft Feet
11 C Celsius
12 F Fahrenheit
13 mi Miles
14 mph Miles per hour
15 rpm Rounds per minute
16 % Percent
17 mB millibars

32 possible units can be defined using the 5 bits available

rainer.
edit: added more units and increased to 5 bits

[MikeB]

You still have an unused bit. How about, make this zero for what you have defined, but if it is a 1 then the other 7 bits could contain further configuration information that could include more units.

Mike.

[rainer]

Yes true. i rearranged the bits above and icnreased to 32 possible units.

If we decide to go this path.
Would it be possible in er9x/opentx?
how manny telemetry fields could realistically be added using this method? ( maybe board dependent... on 9x stock just <n> and on better boards a bigger amount?

for the oXv, i would like as a first step the following fields:

-consumed capacity as a 16 bit unsigned value:
- 0..65535 mAh
-current as as 16 bit value as one of the following: ( depending on the used current sensor)
- 0..65535mA
- 0..653.35 A
- 0..65.355 A
-air pressure as 32 bit value
- 0..1999.99 mbar
- vcc voltage of the used arduino (could be used to monitor bec voltage)
+ some values to output some special User interface numbers... used for a setup/programming mode ( e.g. current menu point, and value ,....) to be changed via 1 channel.

but i have more data available like min/max values for most measured values. These might be interesting under some círcumstances as they are stored in the eeprom of the oXs, so they are persistiant over multiple flights.

rainer.

[MikeB]

Slight aside, I wondered about the consumed capacity. Do you really need 65Ah? I was thinking put this in the FUEL field, with the MSb set and the remaining 15 bits as 0-32767 mAh. This would be easy to detect instead of 0, 25, 50, 75 and 100 of the FrSky fuel sensor.

Mike.

[jhsa]

Mike what about someone with a real fuel sensor? I mean with an IC engine.. Could we still use the fuel field?

Rainer, above I think you forgot a feature already present on the oXs.. You mentioned the BEC voltage but not the pack voltage which we can also read and send down to the tx..

Also, a nice GUI for the sensor would be the icing on the cake

João

[MikeB]

Of course, if the MSbit is clear, then it is a real fuel value, if it is set, then it is a mAh value.

Mike.

[jhsa]

Good, thanks..

[rainer]

Slight aside, I wondered about the consumed capacity. Do you really need 65Ah? I was thinking put this in the FUEL field, with the MSb set and the remaining 15 bits as 0-32767 mAh. This would be easy to detect instead of 0, 25, 50, 75 and 100 of the FrSky fuel sensor.

Mike.

i am actually transferring it in 2 fields.
one would be the new field containing a mAh count, the other one is already the fuel field as a % value related to a definable "max capacity to be used".
So for one of my gliders i configured 4000 as the capacity limit in the oXs and i get a % value showing me how full my battery still is in the fuel field. Additionally i can read how many mAh have been consumed in the new field.
rainer