OpenTX 2.0.12/Taranis - critical bug or defective hardware?

Post Reply
CapnSwash
Posts: 13
Joined: Tue Aug 19, 2014 9:23 pm
Country: -

OpenTX 2.0.12/Taranis - critical bug or defective hardware?

Post by CapnSwash »

Good evening...

After waiting for weeks for my new Taranis B01 (Mode 2) I finally got it a few days ago. I currently have an LRP UpStream 1400 (Bixler clone) equipped with an X6R, FLVSS, GPS and Vario; the usual thing - first fly a testship.

Today I have had a strange thing happen to me for the second time...

When I started, Logical switch 1 was configured as "(S1 > 80) AND SF up".
During the flight it reconfigured itself to "(S1 > 80) AND !SA up"

I noticed the difference only during landing preparation when the motor didn't behave like it should. I happen to have *.eepes before and after, so I am not halucinating.
As mentioned this has happened to me before (the exact same thing) but at the time that was my first flight and I attributed it to my fault during programming. I do not recall which exact version that was but I think it was 2.0.10 WITH compiler trouble, which had its own problems...

My model configuration can be seen in the attached *.eepes (the version with _AF was read from the radio after the flight, the other one is the one the went into the radio yesterday evening. The model in question is model 1). My current version is Bootlaoder 2.0.12 and OpenTx 2.0.12 / 2014-09-16 / 23:30:05 / EEPR: 216; Options ppmus and lua. Never mind the difference in trainer setup - that was wanted.

I am using the example telemetry screens from the homepage with this model. The flight was logged, with 2.0.10 I once had the "SD-Card error" happen to me. This flight was (other than L1) calm with nothing special to remark.

Obviously I don't appreciate this happening in flight... luckily I understood the new configuration for L1 and was able to safely land nonetheless.

So the questions to the experts:
  • Is this a bug in OpenTX?
  • Am I missing something?
  • Or do I have a broken radio (defective RAM maybe)?
  • Is there a hardware test firmware I could use to test the radio?
Regards, Otto

P.S.: I don't think it matters, as this happened in the radio, but I am using Companion 2.0.12 on Win7 32 bit.
Attachments
TaranisSOH_R4.eepe
eepe BEFORE flight.
(77.03 KiB) Downloaded 252 times
TaranisSOH_R4.eepe
eepe AFTER flight and spontaneous configuration change.
(77.03 KiB) Downloaded 234 times

bertrand35
9x Developer
Posts: 2764
Joined: Fri Dec 30, 2011 11:11 pm
Country: -

Re: OpenTX 2.0.12/Taranis - critical bug or defective hardwa

Post by bertrand35 »

I would vote for a firmware bug. Now we really need that you try to reproduce it on the ground. Could it be a defective audio prompt played? Do you have a Lua script running?
bertrand35
9x Developer
Posts: 2764
Joined: Fri Dec 30, 2011 11:11 pm
Country: -

Re: OpenTX 2.0.12/Taranis - critical bug or defective hardwa

Post by bertrand35 »

Another idea, which screen was displayed when you were flying? This AND field was not blinking (and changed with the AUTOSWITCH)?
CapnSwash
Posts: 13
Joined: Tue Aug 19, 2014 9:23 pm
Country: -

Re: OpenTX 2.0.12/Taranis - critical bug or defective hardwa

Post by CapnSwash »

I will be out for the afternoon but try to reproduce it tonight if possible.

I think we can exclude the Autoswitch scenario - I necer entered the programming screens during the flight. I was using the normal in-flight displays and the telemetry screens. That includes the example lua scripts from the SD card zip.

For the audio files i can try that later - in the eepe you can see the what i am calling. I have english firmware and german audio - need to check whether standard or katrin.

Thanks for the prompt answer! Otto
CapnSwash
Posts: 13
Joined: Tue Aug 19, 2014 9:23 pm
Country: -

Re: OpenTX 2.0.12/Taranis - critical bug or defective hardwa

Post by CapnSwash »

I am using the standard German sounds... I have just played them all and simulated three flights (no receiving end = model; no telemetry coming back). All the normal stuff - motor on, motor off, different announcements, different model configurations - start, thermals, landings. Flight resets, next flight.

No luck so far. I guess I need the model and/or just fly... Just a thought - The change happened on the second flight yesterday. I didn't have a GPS fix during the first flight but waited for one before the second flight.

Is there any way to replay a telemetry log as a simulated flight in the radio?

Cheers, Otto

bertrand35
9x Developer
Posts: 2764
Joined: Fri Dec 30, 2011 11:11 pm
Country: -

Re: OpenTX 2.0.12/Taranis - critical bug or defective hardwa

Post by bertrand35 »

No it's not possible to 'replay' telemetry values. But please tell us if you succeed to reproduce this problem (with the GPS started?), I have really no clue!
CapnSwash
Posts: 13
Joined: Tue Aug 19, 2014 9:23 pm
Country: -

Re: OpenTX 2.0.12/Taranis - critical bug or defective hardwa

Post by CapnSwash »

Good evening...
No it's not possible to 'replay' telemetry values.
. Too bad - that could be really useful in this situation.

Here is what I have found so far
I have not been able to fly the model again since last weekend. I have simulated two flights on the ground with all activities of a normal flight (battery change, flight reset, climb, soar, landing, landing abort) with and without GPS fix and no luck - probably it won't happen again as long as I am watching for it :evil:

There is however another thing I have found, which I can reproduce (even in Companion) and which may or may not be related:

If you take a look at my input and Throttle/aileron mix configuration, you will notice that for FM0 and FM1 I use the throttle stick from -100 to 100 as regular throttle for the motor on channel 1.
For FM2 however, I reconfigure the throttle stick with two inputs lines (1 and 5) to be airbrakes from 0 to -100 and motor from 0 to 100. For each half of the stick there is a curve to create a deadband around center.
If I switch to FM2 and set airbrakes, then switch back to FM0 and put the stick above 0, then switch back to FM2 (quite common with bad landing approaches with me), the airbrakes come back to the last position from last time, FM2 was active, although the output of I5 (and the resulting additive mix on the ailerons) should be 0 in that throttle position.

At the same time throttle does something similar of course although the throttle mix is different, so I get a different result.

The fix for this behaviour is to take out the "Stick side" configuration from I5 and the second line in I1- with the curve it is not really needed but I had it in as double safety in case I ever fat-finger something in the curves.

My interpretation is that the input lines simply don't get processed as long as the stick is on the other side from what it is configured to look at - probably intended. Nonetheless I didn't quite expect that result. I expected the output of the line to be 0 instead of undefined.
And as it is related to my motor killswitch logic, I thought I should mention it.

Last thought: Is it even possible with AUTOSWITCH in programming to generate a switch setting that is "!SA up"? My experiments always result in the positive logic - I have not been able to produce a "!" switch setting with auto switch selection. May be another indication that what happened to me was not unintended programming.

Regards, Otto
User avatar
Kilrah
Posts: 11108
Joined: Sat Feb 18, 2012 6:56 pm
Country: Switzerland

OpenTX 2.0.12/Taranis - critical bug or defective hardware?

Post by Kilrah »

Yes stick side activates the line only on that side, if no line is active on an input the current value is maintained.
For the autoswitch thing, move the switch to SAup, then press +/- together to invert.
CapnSwash
Posts: 13
Joined: Tue Aug 19, 2014 9:23 pm
Country: -

Re: OpenTX 2.0.12/Taranis - critical bug or defective hardwa

Post by CapnSwash »

@Kilrah: Thanks for the confirmation - my bad with the stick sides, then. Although I have to admit that I don't quite see the scenario in which I would use this over a curve, then.
For the inversion of the switch in the radio during autoswitch selection - that isn't even mentioned in the fine German manual by Helle - and I think we can really exclude that I accidentally did that in flight. Especially after today's experience.

Soooo....

I have been unsuccessfully trying to provoke the event on the ground. With and without model link up, logging, non-logging, idling, and me playing with announcements and telemetry screens. Nothing. So eventually I figured the model needs to fly and I went flying today.

It DID happen again. :o :!:

Since this time I was half waiting for it, I tried to pay attention to the scenario:
  • I was in flight mode 0
  • Vario and height announcements were playing (and had been playing for a while - by the time it happened, every relevant numerical value probably had been announced for height already once)
  • The sound broke off in mid-value and the speaker made a buzzing sound (different from the normal background buzz)
  • As far as I remember (70% certainty) I was in the second example LUA telemetry screen (telem2.lua), but after the event I was in the standard model start screen.
    I am 1000% sure I was NOT in any programming screen, though. I really didn't touch any key during the flight other than the "ENT" in the next step. I just used the switches.
  • There was an "SD-Card error" message on the screen, that wanted a confirmation with the "ENT" key - this also happened on the first instance but I don't recall it happening on the second instance after which I wrote the initial post to this thread. Possible I just clicked it away, though.
  • After the event L1 was redefined as described in the initial post (I immediately checked - it is kind of comforting that at least the same thing keeps happening reproducibly)
  • In the Logfile, there is a chunk of max 9 minutes missing and the logging resumes in mid-flight - recognizeable by the sudden jump in height. So the height reference was reset in the vario when I changed the battery, thus the log resumes with >100 above ground.
  • This was the second battery of the day - The radio was on through the battery change and I did a flight reset to reset the clocks after changing the battery
There really was nothing special going on; other than I was getting thermal lift with a Bixler with very bad center of gravity... quite unusual but funny enough the same situation as on 9/19. RSSI was good, no perceived loss of control over the model. The model clearly was under my control even during the period when the error message was on the screen (I reset it with the model in the air)

Today's logfile is inconspicuous - I have created longer logs before, logging after the event resumed at line 13145 and at 138.95m barometric (still climbing) - far enough away from any "magic" binary number. I can post it if required. I just noticed that the clock in the radio is one day behind, so radio timestamp and GPS timestamp are one day apart.
I cannot find a missing bit in the log of 9/19 - so maybe there really was no "SD-Card error" message that day.

So there you have it... I will of course examine the SD-Card but even if it should be bad, an error like this should not reprogram the model memory...

Confused,

Otto
User avatar
Kilrah
Posts: 11108
Joined: Sat Feb 18, 2012 6:56 pm
Country: Switzerland

Re: OpenTX 2.0.12/Taranis - critical bug or defective hardwa

Post by Kilrah »

I would try formatting and reloading the card, and even more impotantly redownload and reflash the firmware. You might have had a corrupted download with unpredictable behavior...

I doubt it really is a firmware bug, as we'd have other reports by now...
CapnSwash
Posts: 13
Joined: Tue Aug 19, 2014 9:23 pm
Country: -

Re: OpenTX 2.0.12/Taranis - critical bug or defective hardwa

Post by CapnSwash »

I will try a different card... The one in the radio is checked (I did a complete testrun before using it) but who knows.

Concerning the corrupted download - The first instance of this happening was with 2.0.11 (not 2.0.10 as stated above - I just re-traced my steps on that computer), the next 2 instances happened with 2.0.12.

After the first consciously witnessed event and opening this thread I downloaded 2.0.12 again and flashed it to the radio again this time using Zadig to exclude any trouble with the SD card. That should make the download corruption very unlikely.

I will fly the model without the LUA scripts present for the next times. Of course it is difficult to prove the absence of a problem by having uneventful flights..
Right now I would rather suspect overall CPU load or filesystem access latency. One culprits could simply be the logging write-rate to the SD card - I currently have it set to 10Hz which produces rather large files if you fly a glider for any length of time (both days the glider had >30 Minutes airtime). I don't know how the used library for filesystem access handles SD-Cards but I could imagine the file handle of the open file getting rather large in case data is appended by copying the file around or if the write strategy is to update the file on disk too often.
Also the controller/card combination may take a while to fulfill write requests if a file gets modified. Flash media (not SSDs) don't make very good harddisk substitues - espcially not with FAT on them. Sequential writing should be OK, though.

Regards, Otto
User avatar
Kilrah
Posts: 11108
Joined: Sat Feb 18, 2012 6:56 pm
Country: Switzerland

Re: OpenTX 2.0.12/Taranis - critical bug or defective hardwa

Post by Kilrah »

OK, with that history the download issue can be skipped indeed.
You can try slowing down logs, it makes no sense logging at 10Hz in the first place as telemetry values usually only arrive at 2-5Hz.
There is an issue that is being worked on where if the SD card returns an error state it is not being reinitialised and thus all subsequent accesses fail, which is probably what you see. But there is no reason for any model setting to be changed even with that.
CapnSwash
Posts: 13
Joined: Tue Aug 19, 2014 9:23 pm
Country: -

Re: OpenTX 2.0.12/Taranis - critical bug or defective hardwa

Post by CapnSwash »

Kilrah wrote:You can try slowing down logs, it makes no sense logging at 10Hz in the first place as telemetry values usually only arrive at 2-5Hz.
There is an issue that is being worked on where if the SD card returns an error state it is not being reinitialised and thus all subsequent accesses fail, which is probably what you see. But there is no reason for any model setting to be changed even with that.
Thanks for the info - I did not quite know with which frequency telemetry gets tranferred... after all this is my testship to put the radio through its paces, So I tried to max everything out (successfully as it seems).

I started reading through the sources of the master branch - quite a lot to take in...

I suppose you are referring to issue 1610? That does seem familiar. Question is how the problem can modify both running model AND *.eepe.

I'll keep trying. Need to prepare the new card (must be kicking around on my desk chaos somewhere). If weather allows, the plane goes flying tomorrow again.

Regards, Otto
CapnSwash
Posts: 13
Joined: Tue Aug 19, 2014 9:23 pm
Country: -

Update Oct. 3 2014

Post by CapnSwash »

Went flying yesterday - same usage profile as usual (Vario, Voice, flight modes), but I deleted the LUA telemetry scripts (the whole model script folder in fact).

NO event. Programming did not change.

Telemetry logging at 10 Hz (to keep the changes made to a controllable number)

One additional change out of necessity: no use of trainer mode (well - I turned it on occasionally but there were no inputs, so no mixer results). My son simply didn't go this time.

So while I wouldn't declare victory yet, this looks like the right trail. Theoretical scenarios from my point of view:
  • LUA problem - maybe buffer overflow? I used the supplied example scripts for altimetry screen and large displays. They are simple enough; I at least didn't see anything critical.
  • General overuse of resources, where use of LUA contributes.
    I.e. 10 Hz logging, multiple sound generators (Vario, telemetry announcements, countdowns), which create a high I/O load for the SD card with concurrent read/write use. Then put LUA on top... BOOM?
I will try to do an ON/OFF test... fly some more without LUA, then turn it back on and see whether I can reproduce the problem (for the time being, I will ignore 2.0.13 which will fix an SD card issue).

If the problem returns, reduce the use of SD card I/O (reduce logging rate or turn logging off altogether, less sounds) and see whether that has an influence.

Regards, Otto
User avatar
dinamich
Posts: 288
Joined: Mon Apr 01, 2013 1:21 pm
Country: Slovenia
Location: Ljubljana

Re: OpenTX 2.0.12/Taranis - critical bug or defective hardwa

Post by dinamich »

This kind of bugs are very hard to find. I hope you can isolate which part of OpenTx is causing you troubles. Please continue this research and keep us updated.
projectkk2glider@github
User avatar
dinamich
Posts: 288
Joined: Mon Apr 01, 2013 1:21 pm
Country: Slovenia
Location: Ljubljana

Re: OpenTX 2.0.12/Taranis - critical bug or defective hardwa

Post by dinamich »

CapnSwash any update on this? We now have a dusscussion aobut this issue on github https://github.com/opentx/opentx/issues/1834
projectkk2glider@github
CapnSwash
Posts: 13
Joined: Tue Aug 19, 2014 9:23 pm
Country: -

Re: OpenTX 2.0.12/Taranis - critical bug or defective hardwa

Post by CapnSwash »

Hi dinamich,

I have seen the issue in GitHub - no news from my side; I have not had an opportunity to fly or rig another test since my last post... personal priorities and unsuitable weather for my testplane (fall in Germany has been too warm, quite beautiful but also too windy for a Bixler).

When I get back to testing, I will directly post results to GitHub.

Regards, Otto

Post Reply

Return to “openTx for FrSky radios”