Page 2 of 5
Re: FrSky X10s
Posted: Mon May 11, 2020 1:51 pm
by Suncoaster
MikeB wrote: ↑Mon May 11, 2020 8:58 am
One other thing, please go to the SD Card menu and let me know what the number is at the right of the line that says "Present Yes". We would like to see 32.
Mike
Yes, that is what is there.
Re: FrSky X10s
Posted: Tue May 12, 2020 12:09 am
by MikeB
This is quite odd. The watchdog should get re-triggered every 10mS or so. It has a timeout of half a second so should not really happen.
One of the two things that needs to occur for the watchdog to be triggered is the pulses to be calculated.
Please could you try setting a channel in "servo test" mode (there is a template for that). If you then have a servo moving while you are trying to edit text, it should either move smoothly if all is OK, or it might stutter if something is stopping the pulse calculation from running.
Mike
Re: FrSky X10s
Posted: Tue May 12, 2020 1:03 pm
by Suncoaster
MikeB wrote: ↑Tue May 12, 2020 12:09 am
This is quite odd. The watchdog should get re-triggered every 10mS or so. It has a timeout of half a second so should not really happen.
One of the two things that needs to occur for the watchdog to be triggered is the pulses to be calculated.
Please could you try setting a channel in "servo test" mode (there is a template for that). If you then have a servo moving while you are trying to edit text, it should either move smoothly if all is OK, or it might stutter if something is stopping the pulse calculation from running.
Mike
I think this is as frustrating for you as it is for me. I have had four different watchdog codes and have not even entered a single character, >86, >98 >102 >112 and all I have been doing is scrolling through the keyboard. It doesn't matter if a servo test is running or not. With the system just sitting at the main screen it has reset with watchdog >112 but the reset was almost not noticeable and the servo test just kept running.
Dave
Edit: I don't know if this is significant or not but when the system comes back from the reset, the voltage on the display starts rather low and very quickly returns to 7.6v.
Re: FrSky X10s
Posted: Tue May 12, 2020 3:21 pm
by MikeB
Please go to the "STAT2" menu and report the values there.
While in the menu, do a short press of the MENU (SYS) button to reset the values, then report.
Mike
Re: FrSky X10s
Posted: Wed May 13, 2020 1:32 pm
by Suncoaster
MikeB wrote: ↑Tue May 12, 2020 3:21 pm
Please go to the "STAT2" menu and report the values there.
While in the menu, do a short press of the MENU (SYS) button to reset the values, then report.
Mike
Hi Mike
This is what I have as per your request, I can attach photos if you want.
Before
On Time 02:11:04
tmain 4.14 ms
tBgRead 0.00 ms
tmixer 0.06 ms 402
Voice underruns 0
[MENU] to refresh
Idle time 74.17%
After
On Time 02:11:16
tmain 2.32 ms
tBgRead 0.00 ms
tmixer 0.06 ms 400
Voice underruns 0
[MENU] to refresh
Idle time 74.17%
One other thing I have noticed is that when the system resets it defaults to model one regardless of the model I was editing.
Dave
Re: FrSky X10s
Posted: Wed May 13, 2020 2:51 pm
by MikeB
It all looks OK thanks.
I'll look into why it goes to model 1.
I'm working on some debug code that will dump a lot of data to a file when the watchdog reboots.It may take me a day or two to get it completely written and tested.
Mike
Re: FrSky X10s
Posted: Thu May 14, 2020 12:46 am
by Suncoaster
MikeB wrote: ↑Wed May 13, 2020 2:51 pm
It all looks OK thanks.
I'll look into why it goes to model 1.
I'm working on some debug code that will dump a lot of data to a file when the watchdog reboots.It may take me a day or two to get it completely written and tested.
Mike
Thanks Mike, there is no rush for this, I am sure you have much more important things on your plate, but it is much appreciated.
Dave
Re: FrSky X10s
Posted: Thu May 14, 2020 5:50 pm
by MikeB
I've posted a new test version. This one stores lots of data in memory while running. If there is a watchdog reboot, then it leaves that data alone on the restart and saves it to a file in the root of the SD card.
If/when you get a watchdog reboot, power off. Then we need the file from the SD card. Either restart in bootloader mode so you can read the file that way or remove the SD card and read it in your PC.
The file is called WdDump.txt, probably best to zip it up for posting.
What this does is store data every 1mS while running, indicating where the processor is executing, and the state of various timers and the heartbeat location that controls triggering the watchdog. It should give me the last 700mS prior to the watchdog reboot.
This may give me a good idea of what happened, or at least a good idea of where else to look.
Mike
Re: FrSky X10s
Posted: Fri May 15, 2020 1:41 pm
by Suncoaster
MikeB wrote: ↑Thu May 14, 2020 5:50 pm
I've posted a new test version. This one stores lots of data in memory while running. If there is a watchdog reboot, then it leaves that data alone on the restart and saves it to a file in the root of the SD card.
If/when you get a watchdog reboot, power off. Then we need the file from the SD card. Either restart in bootloader mode so you can read the file that way or remove the SD card and read it in your PC.
The file is called WdDump.txt, probably best to zip it up for posting.
What this does is store data every 1mS while running, indicating where the processor is executing, and the state of various timers and the heartbeat location that controls triggering the watchdog. It should give me the last 700mS prior to the watchdog reboot.
This may give me a good idea of what happened, or at least a good idea of where else to look.
Mike
Hi Mike it was when, dump attached.
Re: FrSky X10s
Posted: Fri May 15, 2020 7:35 pm
by MikeB
A useful dump, it has given me a good idea where to look.
I've added some more items to that dump. I've also found an error in how I'm clearing an interrupt flag in the code that write to the display, although I'm not sure it is causing the problem, however I've fixed it. I've done a couple of other changes that might help.
Please try the test version I've just posted and post the WdDump files if/when(!) it reboots.
Mike
Re: FrSky X10s
Posted: Sat May 16, 2020 1:20 pm
by Suncoaster
Hi Mike,
It is a little better but not much. Dump attached.
One thing I noticed was the date stamp on the dump 01 Jan 2048. The date on both the radio and my computer are correct.
Dave
Re: FrSky X10s
Posted: Sat May 16, 2020 3:39 pm
by MikeB
New test version posted. I can see the special graphic DMA that writes characters and lines to the display is failing to complete operations.
Why is another question, however, I've put in code to re-initialise both the display hardware and the SDRAM hardware when the 0.5 second timeout occurs. If that doesn't get things back, then there should be a watchdog reboot after 5 seconds.
If hardware is re-initialised then you should get an ALERT displayed.
Mike
Re: FrSky X10s
Posted: Sun May 17, 2020 1:41 pm
by Suncoaster
Next dump. About 5 seconds before it rebooted. Is there any significance with the date stamp on the dump file.
Dave
Re: FrSky X10s
Posted: Sun May 17, 2020 2:48 pm
by MikeB
I assume you didn't see a LCD RESET alert message.
I need to re-write what gets dumped to get some more information as to what is causing the graphic DMA to fail to complete.
The data stamp on the file is probably caused by writing the file before I've read the date-time from the RTC chip.
Mike
Re: FrSky X10s
Posted: Sun May 17, 2020 2:55 pm
by Suncoaster
MikeB wrote: ↑Sun May 17, 2020 2:48 pm
I assume you didn't see a LCD RESET alert message.
I need to re-write what gets dumped to get some more information as to what is causing the graphic DMA to fail to complete.
The data stamp on the file is probably caused by writing the file before I've read the date-time from the RTC chip.
Mike
No I did not see any message, the system just reset.
Dave
Re: FrSky X10s
Posted: Sun May 17, 2020 11:00 pm
by MikeB
I'll be a day or so before I have the next test version ready. I do have some code that seems to be working in eepskye that reads and writes the settings and models on the X10. I've also checked many operations of eepskye to make sure it recognises the X10.
Mike
Re: FrSky X10s
Posted: Mon May 18, 2020 1:15 am
by Suncoaster
Thanks Mike, that will be good and I am not in a great hurry, but it will be great to get flying with the X10.
Dave
Re: FrSky X10s
Posted: Mon May 18, 2020 11:00 pm
by MikeB
I've posted a new version for you to test.
I can see the graphic DMA is hanging, but I don't know why. I've changed the debug dump to try to provide more information. I wish I could make this problem happen on my X10Sexpress!
Mike
Re: FrSky X10s
Posted: Tue May 19, 2020 12:45 pm
by Suncoaster
MikeB wrote: ↑Mon May 18, 2020 11:00 pm
I've posted a new version for you to test.
I can see the graphic DMA is hanging, but I don't know why. I've changed the debug dump to try to provide more information. I wish I could make this problem happen on my X10Sexpress!
Mike
Dump attached. This version was a bit more stable it allowed about 6 character input before it reset.
Dave
Re: FrSky X10s
Posted: Tue May 19, 2020 10:40 pm
by MikeB
I cannot see why the graphic DMA is locking up, I reset it before every operation, and also if it fails to complete.
Clearly a restart gets it working again!
I've posted a new (tenth) test version. For this one I've significantly reduced the timeout (1.5mS instead of 16mS) for the graphic DMA. This may stop the watchdog from triggering, but may cause the display to stop updating. If the display does stop updating, the buttons and encoder should still work.
Mike
Re: FrSky X10s
Posted: Wed May 20, 2020 6:01 am
by Suncoaster
MikeB wrote: ↑Tue May 19, 2020 10:40 pm
I cannot see why the graphic DMA is locking up, I reset it before every operation, and also if it fails to complete.
Clearly a restart gets it working again!
I've posted a new (tenth) test version. For this one I've significantly reduced the timeout (1.5mS instead of 16mS) for the graphic DMA. This may stop the watchdog from triggering, but may cause the display to stop updating. If the display does stop updating, the buttons and encoder should still work.
Mike
Reset after approximately 5 seconds. Dump attached.
Re: FrSky X10s
Posted: Wed May 20, 2020 11:06 am
by MikeB
Well it is different, and now unclear why the watchdog triggered. I'll need to enhance the report further.
When you say it reset after 5 seconds, was this 5 seconds after power on, or did the display freeze for 5 seconds first?
I've just noticed that some interrupt handlers (mainly specific to the X10) are not reporting they are running, so I'll add reports to those.
I'll post a new test version later today.
Mike
Re: FrSky X10s
Posted: Wed May 20, 2020 1:34 pm
by Suncoaster
MikeB wrote: ↑Wed May 20, 2020 11:06 am
Well it is different, and now unclear why the watchdog triggered. I'll need to enhance the report further.
When you say it reset after 5 seconds, was this 5 seconds after power on, or did the display freeze for 5 seconds first?
I've just noticed that some interrupt handlers (mainly specific to the X10) are not reporting they are running, so I'll add reports to those.
I'll post a new test version later today.
Mike
Sorry I should have expressed myself more clearly, it was about 5 seconds after I started entering data using the keyboard, in the General menu.
Re: FrSky X10s
Posted: Wed May 20, 2020 9:29 pm
by MikeB
Another test version posted, this problem is really obscure!
If you could try this several times and so get a few dump files it may help see a pattern in the reboots.
Mike
Re: FrSky X10s
Posted: Thu May 21, 2020 1:55 pm
by Suncoaster
MikeB wrote: ↑Wed May 20, 2020 9:29 pm
Another test version posted, this problem is really obscure!
If you could try this several times and so get a few dump files it may help see a pattern in the reboots.
Mike
I am setting up a new Model 4, I was in the Protocol menu I turned on the internal module, and was in the process of changing the Rx number from 2 to 4.
In the General menu starting to change the Model name and I had only entered one character.
Just these three for the moment I will continue testing other operations. I noticed that after each reset the radio is reverting to Model one. This last one was again trying to change the model name and I managed to add five characters before the reset.
I was able to complete most other functions, calibration, template, mixer, voice/audio safety switches, all the functions that you would normally do in setting up a new model.
Dave
Re: FrSky X10s
Posted: Thu May 21, 2020 11:20 pm
by MikeB
Reverting to Model 1 after a reboot suggests the SD card doesn't read properly, although it has already written the dump file!
As a test, could you try running without the SD card in. This will clearly operate with default data, and nothing will be saved but it will be interesting to see if you get a reboot.
I've got to add more debug information to the dump file, the latest dumps are showing the software watchdog is not reaching the expected 500 value before everything stops and the the hardware watchdog must cause the reboot. This may take a day or two to work out what I need to add.
Mike
Re: FrSky X10s
Posted: Fri May 22, 2020 1:22 pm
by Suncoaster
MikeB wrote: ↑Thu May 21, 2020 11:20 pm
Reverting to Model 1 after a reboot suggests the SD card doesn't read properly, although it has already written the dump file!
As a test, could you try running without the SD card in. This will clearly operate with default data, and nothing will be saved but it will be interesting to see if you get a reboot.
I've got to add more debug information to the dump file, the latest dumps are showing the software watchdog is not reaching the expected 500 value before everything stops and the the hardware watchdog must cause the reboot. This may take a day or two to work out what I need to add.
Mike
Mike
I removed the SD card and was able to do everything except use the keyboard. I setup a model, adding safety switches, voice alarms, templates, mixer editing, calibration etc without a problem, only when I went to add/edit text did the system reset.
Dave
Re: FrSky X10s
Posted: Fri May 22, 2020 2:57 pm
by MikeB
It's something to do with the amount of drawing being done on a screen image. I've just managed to make my X10S express fail in the same way by drawing a lot more on the text editing screen. It takes a while, but eventually it I got a watchdog reboot. I've now seen this happen 3 times, so hopefully I'll be able to find what is causing the lockup. It may take me a few days to find, so I'll post progress here.
Mike
Re: FrSky X10s
Posted: Fri May 22, 2020 3:18 pm
by Suncoaster
OK thanks Mike I will keep a watch on this thread.
Dave
Re: FrSky X10s
Posted: Sat May 23, 2020 2:38 pm
by MikeB
I think I've found the bug(s). One problem was register bit naming in a STM supplied header file, causing an incorrect bit to be cleared.
I have also found the graphic DMA device very occasionally appears to generate a "complete" interrupt when it hasn't actually finished. I can't find any reason for this to happen, but I can detect it occurring and avoid any resulting problems.
I'll post a new test version later today.
New test version now posted.
Mike