Home
JAQForum Ver 20.06
Log In or Join  
Active Topics
Local Time 15:00 02 May 2024 Privacy Policy
Jump to

Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me.

Forum Index : Microcontroller and PC projects : MMB4L: MMBasic for Linux alpha release

     Page 9 of 11    
Author Message
thwill

Guru

Joined: 16/09/2019
Location: United Kingdom
Posts: 3848
Posted: 01:51pm 18 Jan 2022
Copy link to clipboard 
Print this post

  JohnS said  The time for the copy is likely irrelevant.

I agree.

  Quote  The speed seems very oddly slow, as if we're overlooking something...

That's my thought too.

The next thing I intend to do in this area is write some minimal pure C tests for serial comms on Linux. That way (a) I will know what the maximum performance is, and (b) if I share that code the other C aficionados (and if necessary the denziens of Stack Overflow) will have an opportunity to point out my mistakes and explain what a doofus I am .

Best wishes,

Tom
Edited 2022-01-18 23:57 by thwill
Game*Mite, CMM2 Welcome Tape, Creaky old text adventures
 
Volhout
Guru

Joined: 05/03/2018
Location: Netherlands
Posts: 3550
Posted: 07:53pm 18 Jan 2022
Copy link to clipboard 
Print this post

Hi Tom,

I tried to test the RTS/CTS function in the MMB4L communication.

Let me explain:
When hardware flow control (RTS/CTS) is used the RTS pin signals that the UART receiver can receive data. When receive buffers are full, the RTS# (TTL level UART pin) goes high, indicating the transmitter to stop sending data.

A simple way to test this is:
On the receiver side, open the com port with RTSCTS and refuse to read any data
On transmitter side, keep sending data unti RTS# goes high. Note how many characters have been sent.
Then allow the receiver to receive data from the buffers, and count the characters.
If both numbers are equal the RTS function works.

I tried this with a CMM2(com1) and MMB4L(/dev/ttyUSB0) using a FTDI TTL interface, both running at 115200 baud. I estimated there would be something like 4096 (your circular buffer) and 1024 (default MMBasic buffer) and 16 (HW UART buffer), so something like 5k characters to fill.

But after running for several minutes, and several Mbyte of data have been sent to the MMB4L com port, RTS has still not changed state.
On MMB4L I can read 4591 characters from the buffers.

So RTS does not work.

These are my test programs:

CMM2
open "com1:115200" as #1

setpin 3,din    'connected to RTS
setpin 5,dout   'connected to CTS
pin(5)=pin(3)   'set default

i=0
'print characters until RTS becomes high
do
 print #1,"@";  
 i=i+1
loop until inkey$=chr$(27) or pin(3)=1
print i;" characters have been sent"


MMB4L
'open com port with hardware handshaking
Open "/dev/ttyUSB0: 115200, RTSCTS" As #1

'do not read any characters until a keyboard key is pressed
do:loop until inkey$<>""

i=0
'now read all characters in the serial buffers and count the characters
do while LOC(1)>0
a$=input$(1,#1)
i=i+1
loop
print i;" characters read"


The wiring:
connect CMM2 TX to RX on PC interface
connect CMM2 RX to TX on PC interface
connect CMM2 GND to GND on PC interface
connect CMM2 pin3 to RTS on PC interface
connect CMM2 pin 5 to CTS on PC interface
The PC interface has TTL level outputs (not +/-12V)
I used TTL-232RG from FTDI.
PicomiteVGA PETSCII ROBOTS
 
thwill

Guru

Joined: 16/09/2019
Location: United Kingdom
Posts: 3848
Posted: 08:41pm 18 Jan 2022
Copy link to clipboard 
Print this post

Thanks Volhout,

  Quote  I estimated there would be something like 4096 (your circular buffer) and 1024 (default MMBasic buffer) and 16 (HW UART buffer), so something like 5k characters to fill.


My ring buffer IS the "default MMBasic buffer", it just happens to default to 4K in MMB4L and 1K in most(?) other MMBasic implementations.

  Quote  But after running for several minutes, and several Mbyte of data have been sent to the MMB4L com port, RTS has still not changed state.
On MMB4L I can read 4591 characters from the buffers.


Sorry, after scratching my head for 15 minutes I realise I've sent you on a wild goose chase. The 4K ring buffer can't be filled, it just throws away the oldest data - which I believe may be standard MMBasic Rx buffer behaviour, I think I copied it from the DOS version. I guess that behaviour will need disabling if the RTSCTS flag is set !!! - presumably that will apply to XONXOFF too ?

Sorry again,

Tom
Edited 2022-01-19 06:45 by thwill
Game*Mite, CMM2 Welcome Tape, Creaky old text adventures
 
lizby
Guru

Joined: 17/05/2016
Location: United States
Posts: 3016
Posted: 09:44pm 18 Jan 2022
Copy link to clipboard 
Print this post

  thwill said  presumably that will apply to XONXOFF too ?


Not sure what best practice should be. If a user MMB4L program sends XOFF after determining that LOC is approaching the size of the buffer, it should expect that some characters may arrive before the sending device responds by ceasing to transmit. These characters should be added to the buffer, at least until LOC equals the buffer size. The user should have the responsibility to make sure that there remains enough space in the buffer to accommodate the sender's response timing.

At that point (LOC=buffersize), I'd be more inclined to throw away the new rather than overwrite the old.

I assume there could be similar timing issues with RTS/CTS. If MMB4L de-asserts CTS, several more bytes could arrive, and should not be thrown away if there is space in the buffer.
PicoMite, Armmite F4, SensorKits, MMBasic Hardware, Games, etc. on fruitoftheshed
 
thwill

Guru

Joined: 16/09/2019
Location: United Kingdom
Posts: 3848
Posted: 11:03pm 18 Jan 2022
Copy link to clipboard 
Print this post

Some bedtime reading has revealed that the Linux kernel adds its own software buffer (4K? For Rx and TX?) between the UART and MMB4L. My suspicion (hope?) is that if I stop reading into the MMB4L Rx ring buffer once it is full (instead of overwriting the oldest data) then the kernel will handle RTS/CTS and XON/XOFF when enabled without further intervention from MMB4L, including dealing with any additional bytes received - though I also read it can be a bit iffy, but that may be a historical issue.

Best wishes,

Tom
Game*Mite, CMM2 Welcome Tape, Creaky old text adventures
 
Volhout
Guru

Joined: 05/03/2018
Location: Netherlands
Posts: 3550
Posted: 07:48am 19 Jan 2022
Copy link to clipboard 
Print this post

Hi Tom,

The circular buffer, which writes over data when full, is the implementation when no flow control is possible.
In both hardware flow control (RTS/CTS) as well as software flow control (XON/XOFF) there is a threshold or trigger level set (i.e. 50% or 75% buffer fullness). This is to compensate for data that has already been in the pipeline from the sender side.
There is another threshold (i.e. 25% or 0% buffer fullness) that can be used te restart the flow of data by toggling the RTS# line.

Basically XON/XOFF works the same, in stead of toggling RTS#, you send a XOFF character. Hardware flow control is more reliable(*), but requires 2 more wires to be connected.

For CTS# you simply look at the status of that line before you put data from the transmit FIFO into the UART. The UART has it's own (classic is 16 byte deep) FIFO, so maximum you send out 16 characters after CTS# went high. (**)

Volhout

(*) XON/XOFF can fail in the situation where the device (slave) input buffer is not empty, so it can not process the XOFF character immediately, only after it has processed the entire input buffer (FIFO). All that time it can be transmitting data that MMB4L must capture in the 50%/25% remaining buffer.  
(**) the size of FIFO depends on the UART. On some embedded UARTS it is only 1 byte in size. On the classical PC UART (the 16450/16550) it is 16 bytes deep.
Edited 2022-01-19 17:51 by Volhout
PicomiteVGA PETSCII ROBOTS
 
robert.rozee
Guru

Joined: 31/12/2012
Location: New Zealand
Posts: 2290
Posted: 11:23am 19 Jan 2022
Copy link to clipboard 
Print this post

perhaps a silly question, but how many real-world cases actually make use of hardware flow control any more, or use XON/XOFF? the only thing i can think of at the moment is where an Arduino will usually have the nRESET pin wired to DTR or RTS via a small capacitor. this allows an attached host to use the appropriate line to reset the Arduino and gain access to the bootloader for uploading new code.

oh, and there is the Pico, where USB data flow back to the host is blocked unless DTR is asserted. but that is more a bug in TinyUSB than anything else!


in the case of MMB4L, MMbasic it is going to be running on a machine with quite considerable resources (compared to a microcontroller), and in Tom's final versions will likely have a massive RxD buffer available (64k, 256k, 1Mb...), that any attached device at the other end of a serial connection is for all practical purposes incapable of overflowing.


all just my opinion...

cheers,
rob   :-)
 
thwill

Guru

Joined: 16/09/2019
Location: United Kingdom
Posts: 3848
Posted: 11:46am 19 Jan 2022
Copy link to clipboard 
Print this post

  Volhout said  The circular buffer, which writes over data when full, is the implementation when no flow control is possible ...


Thanks for the background Volhout.

I think/hope this will be simple to resolve.

There are at least 3 buffers involved:

UART (16 bytes) <--> Linux kernel (4K ?) <--> MMB4L (user defined, default 4K)

My educated guess is that all the hardware and software flow-control is already built into the Linux kernel (otherwise why does termios provide the flags) and that it can handle all the threshold business and starting and stopping the flow that you describe.

However at the moment that doesn't work because MMB4L is pulling the data from the kernel buffer into its local buffer as fast as it can so the kernel buffer thresholds are never reached and the flow-control logic in the kernel is never exercised. I just need to disable MMB4L "greediness" when the flow-control flags are set, and with a bit of luck it "will just work" (TM). Note that there there are a whole slew of status flags (overflow, parity error, ...) to do with serial comms that in theory could be exposed from the Linux kernel to the MMB4L user, but to be honest I don't see that ever being implemented.

  robert.rozee said  oh, and there is the Pico, where USB data flow back to the host is blocked unless DTR is asserted. but that is more a bug in TinyUSB than anything else!


I haven't seen this Rob, and the "Linux Serial HOWTO" even indicates that Linux doesn't provide general support for DTR/DSR, https://tldp.org/HOWTO/Serial-HOWTO.html#toc19.4.

I have an MMB4L program that uses the alpha 3 serial code to connect to the Pico USB console with no such jiggery-pokery* and allows me to drive the PicoMite console and transfer files (AUTOSAVE and XMODEM) just fine.

* OK, not quite no jiggery-pokery, when I am using MMB4L on one of my Linux VMs the first attempt to connect always fails; it seems the Pico receives no data, but the second identical attempt to connect succeeds. When I'm using MMB4L on a Raspberry Pi to connect I have no such difficulties. <shrugs shoulders>

Best wishes,

Tom
Edited 2022-01-19 21:47 by thwill
Game*Mite, CMM2 Welcome Tape, Creaky old text adventures
 
robert.rozee
Guru

Joined: 31/12/2012
Location: New Zealand
Posts: 2290
Posted: 01:16pm 19 Jan 2022
Copy link to clipboard 
Print this post

  thwill said  I haven't seen this Rob, and the "Linux Serial HOWTO" even indicates that Linux doesn't provide general support for DTR/DSR, https://tldp.org/HOWTO/Serial-HOWTO.html#toc19.4.


this is the code i use (both win32 and Linux versions included):
   case Key of            // alt key functions...
{$IFDEF WINDOWS}
               ord('1'):if CONNECTED=2 then try EscapeCommFunction(SerialHandle, SETDTR) except end;          // set DTR
               ord('2'):if CONNECTED=2 then try EscapeCommFunction(SerialHandle, CLRDTR) except end;          // clear DTR
               ord('3'):if CONNECTED=2 then try EscapeCommFunction(SerialHandle, SETRTS) except end;          // set RTS
               ord('4'):if CONNECTED=2 then try EscapeCommFunction(SerialHandle, CLRRTS) except end;          // clesr RTS
{$ELSE}
               ord('1'):if CONNECTED=2 then try I:=TIOCM_DTR; FpIOCtl(SerialHandle, TIOCMBIS, @I) except end;         // set DTR
               ord('2'):if CONNECTED=2 then try I:=TIOCM_DTR; FpIOCtl(SerialHandle, TIOCMBIC, @I) except end;         // clear DTR
               ord('3'):if CONNECTED=2 then try I:=TIOCM_RTS; FpIOCtl(SerialHandle, TIOCMBIS, @I) except end;         // set RTS
               ord('4'):if CONNECTED=2 then try I:=TIOCM_RTS; FpIOCtl(SerialHandle, TIOCMBIC, @I) except end;         // clesr RTS
{$ENDIF}


FpIOCtl is just the Lazarus wrapper for Linux's ioctl. try clearing DTR and you'll find your Pico becomes unresponsive until DTR is set again. under windows, DTR defaults to being cleared, which meant that all versions of GFXterm for win32 prior to a few months ago (when i added forced-DTR on connect) required windows users to press alt-1 before a Pico would talk to it. Linux didn't have this issue.

unfortunately the Pico community has little time for windows users, and the TinyUSB community has little time for the Pico, hence the root cause of the problem remains unresolved.


  thwill said  I have an MMB4L program that uses the alpha 3 serial code to connect to the Pico USB console with no such jiggery-pokery* and allows me to drive the PicoMite console and transfer files (AUTOSAVE and XMODEM) just fine.

* OK, not quite no jiggery-pokery, when I am using MMB4L on one of my Linux VMs the first attempt to connect always fails; it seems the Pico receives no data, but the second identical attempt to connect succeeds. When I'm using MMB4L on a Raspberry Pi to connect I have no such difficulties. <shrugs shoulders>


hmmm, i don't recall ever having that particular problem, but the core functions in GFXterm's serial code had most of the serial setup bugs shaken out well before the Pico appeared on the scene.


cheers,
rob   :-)
Edited 2022-01-19 23:19 by robert.rozee
 
thwill

Guru

Joined: 16/09/2019
Location: United Kingdom
Posts: 3848
Posted: 02:09pm 19 Jan 2022
Copy link to clipboard 
Print this post

  robert.rozee said  ... Linux didn't have this issue.


OK, explains why I didn't see it then. From https://unix.stackexchange.com/questions/446088/how-to-prevent-dtr-on-open-for-cdc-acm: "When a userland process is opening a serial device like /dev/ttyS0 or /dev/ttyACM0, linux will raise the DTR/RTS lines by default, and will drop them when closing it."

  robert.rozee said  hmmm, i don't recall ever having that particular problem, but the core functions in GFXterm's serial code had most of the serial setup bugs shaken out well before the Pico appeared on the scene.


Thanks anyway, hopefully this one will eventually "come out in the wash", or I will work around it by adding some code to automatically retry the connection.

Best wishes,

Tom
Game*Mite, CMM2 Welcome Tape, Creaky old text adventures
 
Volhout
Guru

Joined: 05/03/2018
Location: Netherlands
Posts: 3550
Posted: 02:20pm 19 Jan 2022
Copy link to clipboard 
Print this post

@Robert,

I am not sure if there is a relation, but Arduino uses the DTR line (from a FTDI or CH340) to reset the ATMEGA328 on UNO (for ages already) and force it back in bootloader mode. This may have become a standard, and the pico follows that standard.

Where in Arduino it is edge triggered, so it always wakes up again, in pico it may be level triggered, so as long as you hold the line, the pico is dead.

Volhout
PicomiteVGA PETSCII ROBOTS
 
thwill

Guru

Joined: 16/09/2019
Location: United Kingdom
Posts: 3848
Posted: 02:44pm 19 Jan 2022
Copy link to clipboard 
Print this post

Hi @mobluse,

  mobluse said  I did not know about envvar, but it works with e.g. ?mm.info$(envvar "PATH").

Yes ... except as I've just discovered because it returns a String there is a limit of 255 characters and your PATH is likely to be longer than that so instead you would use the SYSTEM command:

Dim ls%(256) ' 2048 characters or so depending on OPTION BASE.
System "printenv PATH", ls%()
LongString Print ls%()


  mobluse said  I did not know about copy, but maybe you can modify the program in memory using poke, and then you need to save it.

You could POKE the program memory, but I wouldn't be amenable to supporting the resulting mess if you did . My TODO list includes adding some opt-out protection for this sort of "nonsense".


  mobluse said  I guess that kill doesn't work with directories in GW-BASIC is probably a safety feature. The goal with GW-BASIC was probably that you should be able to do everything in it.

I'll need to take another look at KILL vs. RMDIR and decide whether we really need the latter or the former should be allowed to do both.


  mobluse said  I use WSL 1 and the issue is also when I run WSL Ubuntu on the local machine in Microsoft Terminal (from Store) or Command Prompt (built-in cmd.exe). It is not so easy to switch between WSL 1 and WSL 2 and I believe I need some features of WSL 1 such as better support for USB serial ports ...

I installed the default WSL on my Windows 10 machine which is WSL2 (5.10.16.3-microsoft-standard-WSL2) and a brief smoke-test shows it all working fine without the issue you reported; I ran a walkthrough of Scott Adams Adventure #1 through SAAINT and also my Mandelbrot example.

Perhaps your issue is always present for WSL1, perhaps it is specific to your setup in some fashion. I'm a little suprised because MMB4L is a "simple" C console program with no dependencies beyond the standard C library (not even ncurses.) In any case I'm afraid it's not something that I have the time to investigate further; as far as I'm concerned it running out of the box in WSL2 is an unexpected bonus.

Thank you for your feedback,

Tom
Edited 2022-01-20 00:58 by thwill
Game*Mite, CMM2 Welcome Tape, Creaky old text adventures
 
TassyJim

Guru

Joined: 07/08/2011
Location: Australia
Posts: 5909
Posted: 07:27pm 19 Jan 2022
Copy link to clipboard 
Print this post

KILL will delete a directory, provided it is empty.

Jim
VK7JH
MMedit   MMBasic Help
 
lizby
Guru

Joined: 17/05/2016
Location: United States
Posts: 3016
Posted: 05:27pm 22 Jan 2022
Copy link to clipboard 
Print this post

I finally managed to get set up to test MMB4L Alpha 3 for serial receive with XON/XOFF.

For reasons I don't understand, I got "bad file descriptor" when trying to use /dev/ttyAMA0, so I put a CH340 usb/serial adaptor on the OTG port, and succeeded using /dev/ttyUSB0.

I had the serial from the Pi ZW connected to an ESP8266 D1 Mini clone running Annex RDS. This is the program on the ESP:

' ser2pi.bas
xon$=Chr$(17): xoff$=Chr$(19)
D0=16:D1=5:D2=4:D3=0:D4=2:D5=14:D6=12:D7=13:D8=15:D9=3:Rx=3:D10=1:Tx=1
SERIAL2.MODE 115200, D5, D6 ' tx, rx
A$="ABCDEFGHIJKLMNOPQRSTUVWXYZ"
ok$=xoff$
onserial2 received2
k=0

do
 if ok$=xon$ then
   print #2,A$
   k=k+1
   wlog k;" ";
 endif
loop

received2:
 pause 100
 ok$=mid$(serial2.input$,1,1)
return

It waits for an XON and then sends "ABCDEFGHIJKLMNOPQRSTUVWXYZ" in a tight loop until it receives XOFF; it then waits for another XON and repeats.

Here is the MMB4L program, modified slightly from the program I used to test XON/XOFF on the CMM2:


' sertest.bas
Option explicit
Dim string xon=Chr$(17),xoff=Chr$(19),ack=Chr$(6),nak=Chr$(21),a$,b$,c$,d$,fn$
Dim integer i%,j%,k%,l%,m%,n%,cksum%,oldloc%,baud%,nChar%,maxbuf%
dim float t!,ttx!,tmr!

baud%=115200 '1843200 ' 115200 230400 460800 1843200
maxbuf%=2600
'Open "/dev/ttyAMA0:"+Str$(baud)+",10000" As #1
Open "/dev/ttyUSB0:"+Str$(baud)+",10000" As #1
fn$="test.txt"
print "Trying to get "+fn$
open fn$ for output as #2
'a$=fn$+xon
a$=xon
print #1,a$;
oldloc%=0
cksum%=0
nChar%=0
timer=0
do while loc(#1) = 0 and timer < 10000: loop
if loc(#1) = 0 then
print "File Not Received: "+fn+"--exiting"
print #1,chr$(27); ' EOT
pause 1000
end
endif
timer=0
t!=0
ttx!=0
tmr!=timer
do
l%=loc(#1)
if l%=0 then ' transmission done
  if cksum%>0 then ' something received
    print #1,str$(nChar%)+","+str$(cksum%)+xoff
    print " Bytes, checksum: "+str$(nChar%)+","+str$(cksum%)+xoff
    exit do
  else
    pause 2000 ' allow more time for start-up
    if loc(#1) = 0 then ' nothing received
      print "File Not Received--exiting"
'        exit do
      print #1,xon;
    endif
  endif
elseif l%>maxbuf% or oldloc%=l% then ' this block is done; xoff and write to SD card
  print #1,xoff;
  ttx!=ttx!+(timer-tmr!):
  print "*";
  do while loc(#1) > 0 ' clear all characters from buffer
    b$=input$(255,#1)
    j%=Len(b$)
    For i%=1 To j%
      c$=Mid$(b$,i%,1)
      cksum%=cksum%+Asc(c$)
      nChar%=nChar%+1
    Next i%
    print #2,b$;
  loop
  print #1,xon;
  tmr!=timer
endif
oldloc%=l%
pause 1000 ' wait a second
loop
t!=timer
print #1,chr$(27); ' EOT
on error skip 1
close #1
on error skip 1
close #2
print nChar%;" Characters; ";t!/1000;" seconds; ";ttx!/1000;" seconds transmitting; checksum: ";cksum%
print int(nChar%/(t!/1000));" bytes per second total; ";int(nChar%/(ttx!/1000));" bytes per second (to transmit)"

It sends XON, waits until 2600 bytes of the buffer is filled, then sends XOFF and writes what's in the buffer, sends XON and repeats until no bytes are received for over a second, after which it terminates.

This is the output:

> run
Trying to get test.txt
*********************** Bytes, checksum: 65411,4770529
65411 Characters;  64.122 seconds;  51.025 seconds transmitting; checksum:  4770529
1020 bytes per second total;  1281 bytes per second (to transmit)
>

Of the 2336 lines sent, I noted errors in about 4 dozen of them, mostly single-character. That's a different issue, and I haven't investigated it, but XON/XOFF appears to be working under software control for serial streams sent to a PiZW running MMB4L.
PicoMite, Armmite F4, SensorKits, MMBasic Hardware, Games, etc. on fruitoftheshed
 
thwill

Guru

Joined: 16/09/2019
Location: United Kingdom
Posts: 3848
Posted: 07:24pm 22 Jan 2022
Copy link to clipboard 
Print this post

Hi Lance,

  lizby said  I finally managed to get set up to test MMB4L Alpha 3 for serial receive with XON/XOFF ...

... very nice, but not actually what I was expecting to be tested . You have *manually* implemented XON/XOFF flow-control on top of the MMB4L serial interface. MMB4L is supposed to have *automatic* XON/XOFF control if you include the "XONXOFF" flag in the comspec ... however like the hardware flow-control it doesn't work because of MMB4L's greedy Rx ring buffer. Assuming there are no other factors then this should be easy to fix once I've built up a new batch of enthusiasm.

  lizby said  For reasons I don't understand, I got "bad file descriptor" when trying to use /dev/ttyAMA0, so I put a CH340 usb/serial adaptor on the OTG port, and succeeded using /dev/ttyUSB0.

I notoriously don't know what I'm talking about, but did you disable Bluetooth first ? My understanding is that by default the PL011 UART is used for the Bluetooth and /dev/ttyAMA0 is connected to the Bluetooth device, which would explain the error, probably some C I/O configuration function is being called with a flag that is invalid for the Bluetooth driver. See https://raspberrypi.stackexchange.com/questions/45570/how-do-i-make-serial-work-on-the-raspberry-pi3-pizerow-pi4-or-later-models/45571.

  lizby said  Of the 2336 lines sent, I noted errors in about 4 dozen of them, mostly single-character. That's a different issue, and I haven't investigated it ...

I haven't personally tested serial on the Pi-Zero, but have successfully transferred tens of thousands of lines of data fault free between a Pi3 (USB) and a PicoMite.

Best wishes,

Tom
Edited 2022-01-23 05:43 by thwill
Game*Mite, CMM2 Welcome Tape, Creaky old text adventures
 
lizby
Guru

Joined: 17/05/2016
Location: United States
Posts: 3016
Posted: 09:58pm 22 Jan 2022
Copy link to clipboard 
Print this post

Tom--I hadn't realized you were implementing XON/XOFF in hardware. For receiving at least, it is so easy to do it in MMBasic using LOC() that I wouldn't have thought it necessary.

I didn't know about disabling Bluetooth for /dev/ttyAMA0 to work for serial I/O.
PicoMite, Armmite F4, SensorKits, MMBasic Hardware, Games, etc. on fruitoftheshed
 
lizby
Guru

Joined: 17/05/2016
Location: United States
Posts: 3016
Posted: 04:35pm 23 Jan 2022
Copy link to clipboard 
Print this post

I hooked up a Picomite instead of the ESP D1 Mini and modified the code to suit (XOFF sent after buffer has over 2600 bytes; XON sent after all bytes are written):
' ser2pi.bas
Dim integer k
xon$=Chr$(17): xoff$=Chr$(19)
A$="ABCDEFGHIJKLMNOPQRSTUVWXYZ"
ok$=xoff$
SetPin 21,uart0tx:SetPin 22,uart0rx ' : Open "com0:9600" As #1
Open "COM0:115200" As #2
k=0

Do
 If ok$=xon$ Then
   Print #2,A$
   k=k+1
   Print k;" ";
   Pause 3
 EndIf
 If Loc(2) > 0 Then
   b$=Input$(1,# 2)
   If b$=xon$ Or b$=xoff$ Then: ok$=b$: EndIf
 EndIf
Loop

And modified the MMB4L Alpha 3 code on the PiZW to read the output file and report errors.
' sertest.bas
Option explicit
Dim string xon=Chr$(17),xoff=Chr$(19),ack=Chr$(6),nak=Chr$(21),a$,b$,c$,d$,fn$
Dim integer i%,j%,k%,l%,m%,n%,cksum%,oldloc%,baud%,nChar%,maxbuf%
dim float t!,ttx!,tmr!

baud%=115200 '1843200 ' 115200 230400 460800 1843200
maxbuf%=2600
'Open "/dev/ttyAMA0:"+Str$(baud)+",10000" As #1
Open "/dev/ttyUSB0:"+Str$(baud)+",10000" As #1
fn$="test.txt"
print "Trying to get "+fn$
open fn$ for output as #2
'a$=fn$+xon
a$=xon
print #1,a$;
oldloc%=0
cksum%=0
nChar%=0
timer=0
do while loc(#1) = 0 and timer < 10000: loop
if loc(#1) = 0 then
print "File Not Received: "+fn+"--exiting"
print #1,chr$(27); ' EOT
pause 1000
end
endif
timer=0
t!=0
ttx!=0
tmr!=timer
do
l%=loc(#1)
if l%=0 then ' transmission done
  if cksum%>0 then ' something received
    print #1,str$(nChar%)+","+str$(cksum%)+xoff
    print " Bytes, checksum: "+str$(nChar%)+","+str$(cksum%)+xoff
    exit do
  else
    pause 2000 ' allow more time for start-up
    if loc(#1) = 0 then ' nothing received
      print "File Not Received--exiting"
'        exit do
      print #1,xon;
    endif
  endif
elseif l%>maxbuf% or oldloc%=l% then ' this block is done; xoff and write to SD card
  print #1,xoff;
  ttx!=ttx!+(timer-tmr!):
  print "*";
  do while loc(#1) > 0 ' clear all characters from buffer
    b$=input$(255,#1)
    j%=Len(b$)
    For i%=1 To j%
      c$=Mid$(b$,i%,1)
      cksum%=cksum%+Asc(c$)
      nChar%=nChar%+1
    Next i%
    print #2,b$;
  loop
  print #1,xon;
  tmr!=timer
endif
oldloc%=l%
pause 1000 ' wait a second
loop
t!=timer
print #1,chr$(27); ' EOT
on error skip 1
close #1
on error skip 1
close #2
print nChar%;" Characters; ";t!/1000;" seconds; ";ttx!/1000;" seconds transmitting; checksum: ";cksum%
print int(nChar%/(t!/1000));" bytes per second total; ";int(nChar%/(ttx!/1000));" bytes per second (to transmit)"
open fn$ for input as #2
i%=0: k%=0
do while not eof(#2)
 line input #2,a$
 inc i
 if a$ <> "ABCDEFGHIJKLMNOPQRSTUVWXYZ" then
   inc k
 endif
loop
print i%;" lines read; ";k%;" errors"
close #2

Without the PAUSE 3 in the Picomite sending code, about 50% of the lines had errors. With the PAUSE 3, none did (with PAUSE 2, 7 did):

> run
Trying to get test.txt
****** Bytes, checksum: 48356,3519626
48356 Characters;  15.895 seconds;  6.004 seconds transmitting; checksum:  3519626
3042 bytes per second total;  8053 bytes per second (to transmit)
1727 lines read;  0 errors

XON/XOFF under mmbasic program control worked.
PicoMite, Armmite F4, SensorKits, MMBasic Hardware, Games, etc. on fruitoftheshed
 
thwill

Guru

Joined: 16/09/2019
Location: United Kingdom
Posts: 3848
Posted: 05:04pm 23 Jan 2022
Copy link to clipboard 
Print this post

Hi Lance,

Thanks for the test cases. Best guess the 4Kb Linux kernel Rx buffer (which you can't monitor with MMB4L) is filling faster than it is being emptied into the larger MMB4L Rx buffer and thus data is being lost before your XOFF is sent and received.

I think you've reached the point of diminishing returns regarding testing this. Unless you want to try fixing it for yourself you will have to wait for me to get off my arse.

Best wishes,

Tom
Game*Mite, CMM2 Welcome Tape, Creaky old text adventures
 
lizby
Guru

Joined: 17/05/2016
Location: United States
Posts: 3016
Posted: 05:28pm 23 Jan 2022
Copy link to clipboard 
Print this post

Tom,

Without the PAUSE 3, data is lost before the XOFF is sent--for about half the lines with no PAUSE, so it seems to me that the PiZW may be too slow to keep up at 115200 (the CMM2 had no trouble at up to 1843200bps). I have a thus-far untouched PiZW2--I'll see if using that makes a difference (if I can find where I put it).

I have yet to encounter any real-life need to use XON/XOFF (or RTS/CTS), so this is all theoretical for me--I won't try to fix anything.
PicoMite, Armmite F4, SensorKits, MMBasic Hardware, Games, etc. on fruitoftheshed
 
lizby
Guru

Joined: 17/05/2016
Location: United States
Posts: 3016
Posted: 11:15pm 23 Jan 2022
Copy link to clipboard 
Print this post

Hmmm, still needed the PAUSE 3 on the Pi-Z2W.
> run
Trying to get test.txt
XON sent  17
****** Bytes, checksum: 49336,3590956
49336 Characters;  10.731 seconds;  6.002 seconds transmitting; checksum:  3590956
4597 bytes per second total;  8219 bytes per second (to transmit)
1762 lines read;  0 errors

The speed is slightly faster for transmission, but much faster for writing, resulting in about a 33% increase in the speed of transmitting and saving the file.

~
Edited 2022-01-24 09:17 by lizby
PicoMite, Armmite F4, SensorKits, MMBasic Hardware, Games, etc. on fruitoftheshed
 
     Page 9 of 11    
Print this page
© JAQ Software 2024