Home
JAQForum Ver 24.01
Log In or Join  
Active Topics
Local Time 14:37 11 Nov 2025 Privacy Policy
Jump to

Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me.

Forum Index : Microcontroller and PC projects : Pi-cromite 5.4.13, TCP client support

     Page 1 of 2    
Author Message
matherp
Guru

Joined: 11/12/2012
Location: United Kingdom
Posts: 10570
Posted: 01:06am 22 Oct 2017
Copy link to clipboard 
Print this post

Please find attached version 5.4.13.

2017-10-22_104321_mmbasicstretch.zip

2017-10-22_104333_mmbasicjessie.zip

2017-10-22_105729_Pi-cromite_Manual.pdf

Changes are:
1. fixed bug in LONGSTRING TRIM
2. fixed bug in SYSTEM command to avoid last line appearing twice
3. changed parsing so that subroutines and functions can be executed from the command line
4. removed ability to run a disk program without using the RUN command
5. implemented TCPIP client capability
6. implemented TRANSMIT NOHEADER for TCPIP socket functionality

The TCP functionality is shown below using the existing OPEN "socket" capability together with the new TCP client. Also below is an example of using the TCP client capability to interrogate api.openweather.org




Option explicit
Option default none
Dim a$,b$,c$
Dim integer i,j,rlen
Dim m%(4000)
Dim MYID$="enter city code here"
Dim APIKEY$="enter your APIKEY here"
'
' get the IP address of openweathermap and open a client connection to it
'
a$=GetIP$("api.openweathermap.org")
TCP client a$,80
'
' construct the request for a weather report and send it
'
b$= "GET /data/2.5/weather?id="+MYID$+"&APPID="+APIKEY$
b$=b$+" HTTP/1.0"+Chr$(13)+Chr$(10)+Chr$(13)+Chr$(10)
TCP send b$
'
'set up a longstring variable and read the return message into it
'
LongString clear m%()
Do
TCP receive c$
If c$<>"" Then LongString append m%(),c$
Loop While c$<>""
'
' trim off the HTTP header
'
LongString trim m%(),LInStr(m%(),"{")-1
rlen=LLen(m%())
'
' process the message, in this case just print it out
'
i=1
Do While rlen>0
If rlen>240 Then
j=240
Else
j=rlen
EndIf
rlen=rlen-j
Print LGetStr$(m%(),i,j);
i=i+j
Loop
Print ""
TCP close







Edited by matherp 2017-10-23
 
MicroBlocks

Guru

Joined: 12/05/2012
Location: Thailand
Posts: 2209
Posted: 06:26am 22 Oct 2017
Copy link to clipboard 
Print this post

Can you specify a port when using OPEN "socket", similar like when opening com port?


Microblocks. Build with logic.
 
matherp
Guru

Joined: 11/12/2012
Location: United Kingdom
Posts: 10570
Posted: 06:32am 22 Oct 2017
Copy link to clipboard 
Print this post

  Quote  Can you specify a port when using OPEN "socket", similar like when opening com port?


That was existing functionality from when I did the HTML work. The port defaults to 80 but can be set with OPTION SOCKET nnn. This is a permanently stored option so should be run at the command prompt before loading a program.
 
MicroBlocks

Guru

Joined: 12/05/2012
Location: Thailand
Posts: 2209
Posted: 06:59am 22 Oct 2017
Copy link to clipboard 
Print this post

That limits it to only one socket. As the open command uses a 'filenumber' it seems it would be possible to open more then one when the port number is part of the open socket command. Any particular reason or limitation why OPTION SOCKET was chosen?

Microblocks. Build with logic.
 
matherp
Guru

Joined: 11/12/2012
Location: United Kingdom
Posts: 10570
Posted: 07:19am 22 Oct 2017
Copy link to clipboard 
Print this post

  Quote  That limits it to only one socket.


This is a special case designed to support a full HTML server. The socket is opened when mmbasic starts and is always in listen mode. It returns a message "no data" if there isn't an OPEN "SOCKET" active. The "file number" is just used to allow the print command to recognise this is output to a socket rather than a file or the console.

mmbasic is inherently single threaded so the expectation is that it is only used as one of TCP server, TCP client, UDP server or UDP client otherwise things start to get too complex. My examples of UDP and TCP are designed to demonstrate the simple use for inter-machine communications with the one exception of the full HTML server functionality in the manual.
 
MicroBlocks

Guru

Joined: 12/05/2012
Location: Thailand
Posts: 2209
Posted: 08:13am 22 Oct 2017
Copy link to clipboard 
Print this post

The underlying system (linux) is not single threaded so it could be possible to start multiple instances of mmbasic. Is it possible for each instance of MMBasic to have their own OPTION collection? Maybe each instance its own directory?
If the OPEN command supports the port number then this is not needed.
The port number could then be an argument when starting mmbasic.




Microblocks. Build with logic.
 
matherp
Guru

Joined: 11/12/2012
Location: United Kingdom
Posts: 10570
Posted: 08:22am 22 Oct 2017
Copy link to clipboard 
Print this post

  Quote  so it could be possible to start multiple instances of mmbasic


It isn't because of the way I/O works with the pigpio library
 
MicroBlocks

Guru

Joined: 12/05/2012
Location: Thailand
Posts: 2209
Posted: 08:52am 22 Oct 2017
Copy link to clipboard 
Print this post

What if the pigpio library is not needed. Is it possible to not use it at all when the application does not require any gpio?
Maybe with a --nogpio flag for example.




Microblocks. Build with logic.
 
matherp
Guru

Joined: 11/12/2012
Location: United Kingdom
Posts: 10570
Posted: 10:16am 22 Oct 2017
Copy link to clipboard 
Print this post

  Quote  What if the pigpio library is not needed. Is it possible to not use it at all when the application does not require any gpio?
Maybe with a --nogpio flag for example.


That's another version - not mine.
 
Grogster

Admin Group

Joined: 31/12/2012
Location: New Zealand
Posts: 9751
Posted: 08:07pm 22 Oct 2017
Copy link to clipboard 
Print this post

Awesome.

@ MB - With the greatest respect: one TCP/IP socket is not impressive enough considering what the MM is? I think this single-socket connection is impressive enough IMHO....
Smoke makes things work. When the smoke gets out, it stops!
 
bigfix
Senior Member

Joined: 20/02/2014
Location: Austria
Posts: 129
Posted: 10:44pm 22 Oct 2017
Copy link to clipboard 
Print this post

As it fits to this discussion:

A possible solution for plain MicroMites without integrated Networking
Not sure how compatible this is with the Picromite - cannot try right now

Pls stick to the original ESPterm Thread about specific details, to avoid cluttering this one
ESPTerm Thread


I guess most people never looked into ESPterm, which I introduced some weeks ago
ESPterm is a special image on an ESP8266 Module, connected to the serial port of a micromite (best 230kbits)
Configure light background, dark foreground, capture F5,F11,F12 - then the MMbasic Editor works nicely

In Version 2.x besides VT100 Web display, some basic Networking got integrated, debug screen, dynamic picture background from external FTP/HTTP server, password security etc...
But you can read yourself at:
ESPterm Github Releases


Here some infos from the internal ESPterm Helpfile about Networking:

This can run intermixed with terminal traffic on the same COM port
It is done with private messages, which are usually ignored by terminals


[Escape] = Chr$27
[Bell] = Chr$7
^ is the plain "Uparrow" Ascii Symbol - not "Control"

Device-to-device Messaging

To send a message to another ESPTerm module or IP Node, use: [Escape]^M;DestIP;message[Bell].

This command sends a POST request to http://<DestIP>/api/v1/msg.
The IP address may be appended by a port, if needed (eg. :8080).
In addition to POST, a GET request can also be used.
In that case, any GET arguments (/api/v1/msg?arguments) will be used instead of the request body.
This is intended for external access when sending POST requests is not convenient.

Each ESPTerm listens for such requests and relays them to UART:
[Escape]^m;SrcIP;L=length;message[Bell],
with length being the byte length of message, as ASCII.

Notice a pattern with the first letter:
capital is always a command, lower case a response.
This is followed with the HTTP commands and any networking commands added in the future.

Example: Node 192.168.0.10 sends a message to 192.168.0.19:
[Escape]^M;192.168.0.19;Hello[Bell].
Node 192.168.0.19 receives [Escape]^m;192.168.0.10;L=5;Hello[Bell] on the UART.

Note that the IP address in the reception message is that of the first node,
thus it can be used to send a message back.

External HTTP requests

To request an external server, use [Escape]^H;method;options;url\nbody[Bell].

method - can be any usual HTTP verb, such as GET, POST, PUT, HEAD.
options - is a comma-separated list of flags and parameters:
H - get response headers
B - get response body
X - ignore the response, return nothing

N=nonce - a custom string that will be added in the options field of the response message.

Use this to keep track of which request a response belongs to.

T=ms - request timeout (default 5000 ms), in milliseconds
L=bytes - limit response length (default 0 = don't limit).
Applies to the head, body, or both combined, depending on the H and B flags
l=bytes - limit the response buffer size (default 5000 B).
This can reduce RAM usage, however it shouldn't be set too small,
as this buffer is used for both headers and the response body.
url - full request URL, including http://. Port may be specified if different from :80,
and GET arguments may be appended to the URL if needed.
body - optional, separated from url by a single line feed character (\n).
This can be used for POST and PUT requests.
Note: the command may be truncated to the maximum total length of 256 characters if too long.
The response has the following format: [Escape]^h;status;options;response[Bell]

status - a HTTP status code, eg. 200 is OK, 404 Not found.
options - similar to those in the request, here describing the response data.

This field can contain comma-separated B, H and L=bytes and N=nonce.
response - the response, as requested.
If both headers and body are received, they will be separated by an empty line
(i.e. \r\n\r\n). Response can be up to several kilobytes long, depending on the L= and l= options.

Example: [Escape]^H;GET;B;http://wtfismyip.com/text[Bell]
- get the body of a web page (wtfismyip.com is a service that sends back your IP address).
A response could be [Escape]^h;200;B,L=11;80.70.60.50[Bell].Edited by bigfix 2017-10-24
 
MicroBlocks

Guru

Joined: 12/05/2012
Location: Thailand
Posts: 2209
Posted: 11:09pm 22 Oct 2017
Copy link to clipboard 
Print this post

  Grogster said   Awesome.

@ MB - With the greatest respect: one TCP/IP socket is not impressive enough considering what the MM is? I think this single-socket connection is impressive enough IMHO....


Sure a single socket is enough. The reason i asked is that mmbasic is running on a linux system which has no problem running hundreds of sockets.Linux also has no problem running multiple instances of a program.
My suggestion was to use the multitasking capabilites of linux to enable multiple instances of mmbasic. You could then have multiple socket running (one in each mmbasic instance). This makes programming easier as not all data has to be processed by a single process.
Also running multiple tiny servers each with their own tiny program and own directory would make programming and maintaining code simpler.
I understand it is the pigpio library that prevents this so until that is solved starting multpile instances of mmbasic is not possible.


Microblocks. Build with logic.
 
Grogster

Admin Group

Joined: 31/12/2012
Location: New Zealand
Posts: 9751
Posted: 10:27am 23 Oct 2017
Copy link to clipboard 
Print this post

Ahhh, gotcha. I was not thinking along those lines.
Smoke makes things work. When the smoke gets out, it stops!
 
bigfix
Senior Member

Joined: 20/02/2014
Location: Austria
Posts: 129
Posted: 10:34am 23 Oct 2017
Copy link to clipboard 
Print this post

Maybe there could be a "Master" MMbasic or some slim shim acting as a scheduler handling Pigpio as a single task and communicating with the other MMbasics ?Edited by bigfix 2017-10-24
 
matherp
Guru

Joined: 11/12/2012
Location: United Kingdom
Posts: 10570
Posted: 11:49am 23 Oct 2017
Copy link to clipboard 
Print this post

  Quote  Maybe there could be a "Master" MMbasic or some slim shim acting as a scheduler handling Pigpio as a single task and communicating with the other MMbasics ?


There could be lots of things but since I won't be programming any on them then it will be up to someone else to do the work. I'm happy to add functionality that makes sense AND that interests me intellectually but since this is just a retirement hobby for me then bluntly its "take it or leave it".
 
MicroBlocks

Guru

Joined: 12/05/2012
Location: Thailand
Posts: 2209
Posted: 07:22pm 23 Oct 2017
Copy link to clipboard 
Print this post

It looks like you only need to skip the gpioInitialise(void) function to be able to start another instance of mmbasic. Of course manipulating the IO pins is then not available, but the rest of mmbasic is. Maybe a small test will determine if that is correct.
It is functionality that makes a lot of sense only not in your world. As it is as good as impossible to have more then one person working on the source the load is unfortunately only on your shoulders.
Edited by MicroBlocks 2017-10-25
Microblocks. Build with logic.
 
MikeO
Senior Member

Joined: 11/09/2011
Location: Australia
Posts: 275
Posted: 12:05am 28 Oct 2017
Copy link to clipboard 
Print this post

Peter,
It looks like there is an issue with ver 5.4.13 to do with Open Socket. I was unable to receive any data using this version and the browser reported "No Data" when I closed the running (web server) program, indicating No Open Socket. I then went back to 5.4.10 and everything works ok.I am running Jessie full GUI on a Pi3 B.
I hope I have not miss-understood some changes in this version?

My code is below.

BTW, I came across a problem in the parser where a page request eg "test.html" would throw an "out of bounds" error in this line (first one in the do - loop
arg$(0,paramcount)=Mid$(a$,startparam,inpos-startparam)
when there was no actual arguments as in the case of a page request.

my remedy was to check for the existence of BOTH a ? and an =
inpos=Instr(a$,"?")
isequal=Instr(a$,"=")
If inpos<>0 and isequal<>0 Then 'found both ? and =



'MMserver
'Michael Ogden
'October 2017
'PicroMite WebServer
'Original web interrupt and webparse code by matherp of "thebackshed.com"
'
'v0.5


option explicit
option default none
Const maxargs = 32
Dim a%(1000),i%
Dim s$,pg$
Dim security$=""
Dim arg$(1,maxargs-1)

'
'Define I/O
setpin 13, dout 'ActLed


pin(13)=1


'

Open "socket,data_arrival,100" As #2

'main program loop
Do

Pause 2000

Loop




'********************************* Web routines *****************************

'data arrival
Sub data_arrival
Local p%=0, t%=0
Local g$
pin(13)=0 'flash the Activity LED
pause 50
pin(13)=1
print "data arrival.."
Do While Not Eof(2) 'read the data from socket
LongString append a%(),Input$(10,2)
Loop
p%=LInStr(a%(),"GET",1) 'check for GET and HTTP
t%=LInStr(a%(),"HTTP",1)
print p%;" ";t%
If p%<>0 And t%<>0 Then 'full request received
s$=LGetStr$(a%(),p%,t%-p%+4)'filter out request string
print s$
LongString trim a%(),t%+4 'trim off used portion, keep the rest in buffer
pg$= parserequest$(s$,i%) 'Parse args from request
If i% Then 'if there are arguments
if security$<>"" then 'if there is a security password
If arg$(0,0)="Security" And arg$(1,0)<>security$ Then 'Not valid update
pg$=""
goto Response
end if
UserArgs
EndIf
EndIf
'
Response:
If pg$="index.html" Then
Transmit page "index.html"
Else
Transmit code 404
EndIf
EndIf
End Sub

sub UserArgs
' If arg$(0,1)="R" Or arg$(0,2)="R" Then
' cp(checked)=""
' If arg$(0,1)="R" Then checked=Asc(arg$(1,1))-Asc("A")
' If arg$(0,2)="R" Then checked=Asc(arg$(1,2))-Asc("A")
' cp(checked)=check$
' ' updateheater
' EndIf
end sub

'Function to parse an HTML GET request'
' Assumes that the request starts with "GET /"
' and ends with "HTTP"
'
Function parserequest$(req$, paramcount As integer)
Local a$,b$
Local integer inpos,startparam,processargs,isequal
For inpos=0 To maxargs-1
arg$(0,inpos)=""
arg$(1,inpos)=""
Next inpos
paramcount=0
a$=Mid$(req$,6,Len(req$)-10) 'starts from char after GET / , ends before HTTP
inpos=Instr(a$,"?")
isequal=Instr(a$,"=")
If inpos<>0 and isequal<>0 Then 'parameters found both ? and =
processargs=1
parserequest$=Left$(a$,inpos-1)
a$=Mid$(a$,inpos+1) 'get rest of line after ?
Do 'loops through recovering args and values
arg$(0,paramcount)=""
arg$(1,paramcount)=""
inpos=Instr(a$,"=")
startparam=1
arg$(0,paramcount)=Mid$(a$,startparam,inpos-startparam)
startparam=inpos+1
inpos=Instr(a$,"&")
If inpos<>0 Then
arg$(1,paramcount)=Mid$(a$,startparam,inpos-startparam)
a$=Mid$(a$,inpos+1)
paramcount=paramcount+1
Else
arg$(1,paramcount)=Mid$(a$,startparam)
paramcount=paramcount+1
processargs=0
EndIf
Loop While processargs
Else
parserequest$=a$
print "parserequest:";a$
EndIf
If a$="" Then
parserequest$="index"
EndIf
If Instr(parserequest$,".html")=0 And Instr(parserequest$,".HTML")=0 Then 'add html if missing
parserequest$=parserequest$+".html"
end if
End Function


Codenquilts
 
matherp
Guru

Joined: 11/12/2012
Location: United Kingdom
Posts: 10570
Posted: 06:39am 28 Oct 2017
Copy link to clipboard 
Print this post

  Quote   I was unable to receive any data using this version and the browser reported "No Data" when I closed the running (web server) program


Ooops Left in a diagnostic change - sorry.

Corrected version for Stretch

2017-10-28_162332_mmbasicstretch.zip

and for Jessie

2017-10-28_163549_mmbasicjessie.zip

This version also includes a development I'm working on to parse JSON strings

JSON$(array%(),string$)

Returns a string representing a specific item out of the JSON input stored in the longstring array%()

e.g.
JSON$(a%(), “name”)
JSON$(a%(), “coord.lat”)
JSON$(a%(), “weather[0].description”)
JSON$(a%(),”list[4].weather[0].description

Examples taken from api.openweathermap.org




Option explicit
Option default none
Dim a$,b$,c$
Dim integer i,j,rlen
Dim m%(4000)
Dim MYID$="2638878"
Dim APIKEY$="Your APIKEY here"
Dim report$="weather"
a$=GetIP$("api.openweathermap.org")
TCP client a$,80
b$= "GET /data/2.5/"+report$+"?id="+MYID$+"&APPID="+APIKEY$
b$=b$+" HTTP/1.0"+Chr$(13)+Chr$(10)+Chr$(13)+Chr$(10)
TCP send b$
LongString clear m%()
Do
TCP receive c$
If c$<>"" Then LongString append m%(),c$
Loop While c$<>""
LongString trim m%(),LInStr(m%(),"{")-1
rlen=LLen(m%())
i=1
Open report$+".json" For output As #1
Do While rlen>0
If rlen>240 Then
j=240
Else
j=rlen
EndIf
rlen=rlen-j
Print #1,LGetStr$(m%(),i,j);
' Print LGetStr$(m%(),i,j);
i=i+j
Loop
Print ""
TCP close
Close #1
Print "Weather for "+json$(m%(),"name")
Print "Temperature is ",val(json$(m%(),"main.temp"))-273
Print "Pressure is ",json$(m%(),"main.pressure")
Print json$(m%(),"weather[0].description")



 
lew247

Guru

Joined: 23/12/2015
Location: United Kingdom
Posts: 1702
Posted: 08:38am 07 Jun 2018
Copy link to clipboard 
Print this post

This is brilliant
Thanks Peter

One question

If the API request uses HTTPS such as DarkSky
Is there a way to get this to work without setting up a PHP server on the Pi and then using that to get and parse the data?

example

[code]https://api.darksky.net/forecast/APIKEY$/53.4399,-2.1065?units=uk2[/code]

When I use the code above I get this result



IF I change the get IP line to
a$=GetIP$("https://api.darksky.net")

Then I get this result



Yet if I do a tracert on https://api.darksky.net I get

 
matherp
Guru

Joined: 11/12/2012
Location: United Kingdom
Posts: 10570
Posted: 09:50am 07 Jun 2018
Copy link to clipboard 
Print this post

GETIP$ won't work with HTTPS addresses. Don't have a solution for this - but try omitting "HTTPS://" and see if you get the correct result - I get 34.194.205.2

You can also try putting the physical IP address into the TCP client command - remember to also use port 443Edited by matherp 2018-06-08
 
     Page 1 of 2    
Print this page
The Back Shed's forum code is written, and hosted, in Australia.
© JAQ Software 2025