Home
JAQForum Ver 20.06
Log In or Join  
Active Topics
Local Time 16:48 27 Jan 2022 Privacy Policy
Jump to

Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me.

Forum Index : Microcontroller and PC projects : ESPTerm - webbased VT100

Author Message
bigfix
Senior Member

Joined: 20/02/2014
Location: Austria
Posts: 107
Posted: 10:04am 10 Sep 2017
Copy link to clipboard 
Print this post

Finally - filling another gap in the MicroMite family




ESPterm is a tiny Webserver running on an ESP8266 which emulates a VT100
ESPterm allows to "webify" any Micromite just with a serial connection and no special Web/Network code at all

This gives you a full character based GUI to the MicroMite with optional touch/mouse support
tiny size, relatively low power
very low costpoint - typically a few $ only
no special SW on the client side - just a browser
no OS patches/maintenance - fits well with MicroMites


It was written by Ondrej Hruska in Prague - I only helped with some requirements & testing & some beer money
Ondrej did an excellent job, documenting everything on Github and having almost no bugs in many iterations

His goal is to make it as compatible as possible with all typical character based Unix applications & VTtest
The main limitation is the tiny RAM in the ESP - leaving about 30kB only for the application & Screen storage
Program space is less of an issue
- so we will get many more functions over time and quite extensive GUI & Help pages


Github ESPTerm


ESPTerm Demo

All the menu/help pages in the demo look exactly the same as the real thing

The current releases you find at
Current ESPTerm Release:

Flashing works with the usual tools, which were discussed multiple times in the Forum already
ESP Flashing



Features:
Almost complete VT102 emulation with some extras from Xterm, eg.
Screen size up to 80x25
(Currently no 132 Char Mode & Double Wide/High due to memory restrictions)

All standard text styles and 16 colors supported
Full UTF-8 support, alternate character sets

Configurable mouse support "XTerm Style" - should also work with touchscreens
XTerm Standard mouse tracking modes configurable from serial port

You can dynamically set screen title, button labels from serial port
up to 5 configurable buttons under the screen for quick commands

Real-time screen update via WebSocket - require somewhat up to date browser
Button to open keyboard on Android - use "hackers keyboard app" for full function keyboard

Text file upload tool with configurable delays and line endings

Built-in extensive help page with basic troubleshooting and command reference

Robust WiFi configuration interface
Static IP, DHCP, channel selection, power
SSID search utility for finding your existing network

(Possible) Futures:
Even more compatibility to VTs
TCP/Telnet mode for Client Terminal Emulators or tools like MMedit
Dynamic Background picture from external Webserver
More color selections
Optimized RAM usage allowing more features


Personal MicroMite experience:

In theory ESPterm supports 4 concurrent sessions - but memory is scarce...
In practice
One window works very well
Two windows may have some startup trouble with browsers filling caches - then work stable
Three windows hardly work - especially when the Help page is open


My MMbasic knowledge is still rather rudimentary
- I did not work much with my Mites since Christmas...
Currently garden/construction work has priority
- so I got only some very basic code

I run it on a ESP01 Version, should also work with ESP12

I found ESPterm works best with 230400 Baud, I got buffer overruns at 38400
To use the MMbasic Editor you should set the background to white, foreground to black


The Commandset is rather rich - all extensions are documented in the help pages
One thing bit me initially - Ondrej often uses "C" notation for characters
\a i.e. means a "Bel" Ascii Character Hex 07
This is documented now in the help page

Example Set Title:
E$ = Chr$(27)
B$ = Chr$(7)
Print E$+"]0;This is a Test"+B$

Example Set Button:
E$ = Chr$(27)
B$ = Chr$(7)
Print E$+"]81;Yes"+B$


Example Mouse Usage:

To enable the Mouse in normal click mode use:
Print E$+"[?9h"
To disable the mouse:
Print E$+"[?9l"
Left upper corner click delivers:
[M!! = Hex 33,33 minus basevalue 32 gives 1,1
Lower right corner click delivers:
[Mp8 = Hex 112,56 minus basevalue 32 gives 80,24
This allows you to use any location on the screen as an input button
- not just the 5 predefined buttons on the bottom of the screen


ESP 8266 Hardware - things to watch

Self destruct - Thermal Runaway bug
ESP Self Destruct

Solution: Ondrej's safe breadboard adapter
Safe Breadboard Adapter

PCB Details for Adapter:
Adapter PCB










 
CircuitGizmos

Guru

Joined: 08/09/2011
Location: United States
Posts: 1411
Posted: 12:00pm 10 Sep 2017
Copy link to clipboard 
Print this post

This ties in perfectly with something I'm doing. Nice!

Does it run well in more than just Chrome?Edited by CircuitGizmos 2017-09-11
Micromites and Maximites! - Beginning Maximite
 
bigfix
Senior Member

Joined: 20/02/2014
Location: Austria
Posts: 107
Posted: 11:30pm 10 Sep 2017
Copy link to clipboard 
Print this post

Browser support besides Chrome ?

I just tested it briefly to make sure

Chrome W7, W10 is best looking and works well

Firefox W7, W10 works well and looks a little "thin" fontwise

Msoft Edge W10 (18 month old - I never do updates on W10 ) is terrible
- but you do not want Edge anyhow

Ipad Safari IOS 7.06 (very old Jailbreak...) works marginally

Android Tablet 6.01 Chrome works well - only the Std Keyboard misses keys
Install "Hackers Keyboard" to get all keys

Ondrej spent a lot of time fiddling with stylesheets - it became a compromise
Either you got gaps betwen linedrawing characters or background lines on the screen
He finally used a Google Font, which Firefox does not have



 
bigfix
Senior Member

Joined: 20/02/2014
Location: Austria
Posts: 107
Posted: 11:35pm 10 Sep 2017
Copy link to clipboard 
Print this post

WLAN Scanning Bug

I just got notified that WLAN scanning does not work

It definitely worked very well with the 0.68 & 0.71 versions
Since that I only reflashed the new versions and it kept the old settings.
I just tried a Wifi scan with 1.01 - does not work

I drop Ondrej a note, he is a very nice and responsive guy
BTW You find his email in the "about" page

One temporary workaround could be:
Flash the 7.1 Version (I just tested the scan)
do all net config settings and then flash the 1.01 over
- it keeps the settings

 
CircuitGizmos

Guru

Joined: 08/09/2011
Location: United States
Posts: 1411
Posted: 06:07am 11 Sep 2017
Copy link to clipboard 
Print this post

  bigfix said   WLAN Scanning Bug

I just got notified that WLAN scanning does not work

It definitely worked very well with the 0.68 & 0.71 versions
Since that I only reflashed the new versions and it kept the old settings.
I just tried a Wifi scan with 1.01 - does not work

I drop Ondrej a note, he is a very nice and responsive guy
BTW You find his email in the "about" page

One temporary workaround could be:
Flash the 7.1 Version (I just tested the scan)
do all net config settings and then flash the 1.01 over
- it keeps the settings



I'll try this firmware when the fix for this comes out.
Micromites and Maximites! - Beginning Maximite
 
bigfix
Senior Member

Joined: 20/02/2014
Location: Austria
Posts: 107
Posted: 10:14am 11 Sep 2017
Copy link to clipboard 
Print this post

In the next internal Beta the WIFI bug is gone already

Ondrej redid the complete frontend to make it friendlier to all Browsers

Memory is much more optimized
256 Color support
Cursor style selection
Full screen option

Expect availability next weekend...



 
robert.rozee
Guru

Joined: 31/12/2012
Location: New Zealand
Posts: 1899
Posted: 01:04pm 11 Sep 2017
Copy link to clipboard 
Print this post

for attaching to a micromite, it would be really handy if a spare I/O pin on the ESP8266 could be assigned to one of the screen button. that pin could then be hooked up to reset on an attached micromite (or other micro) for remotely resetting mmbasic.


cheers,
rob :-)
 
bigfix
Senior Member

Joined: 20/02/2014
Location: Austria
Posts: 107
Posted: 08:32pm 11 Sep 2017
Copy link to clipboard 
Print this post

Pin "toggling" was one of my original wishes
besides having a Telnet/TCP mode without VT functions

It may come with an ESP12 only, which has more pins
The ESP01 is rather restricted on I/O


Currently Ondrej has a ton of open issues and works through

This little thing is a rather complex beast in reality
i.e. we got some fine vertical lines on bright backgrounds with the internal beta
This may be due to rounding errors in the renderer - or something else...
 
MicroBlocks

Guru

Joined: 12/05/2012
Location: Thailand
Posts: 2209
Posted: 12:41am 12 Sep 2017
Copy link to clipboard 
Print this post

  bigfix said   Browser support besides Chrome ?

Msoft Edge W10 (18 month old - I never do updates on W10 ) is terrible
- but you do not want Edge anyhow

If you are on W10 you might want to reconsider, you can always roll back an update if you are not satisfied with it. 18 months old in todays terms is like pre historic. Testing stuff on 18 months old OS that was a major new release is not the best way to go.
Lots has changed. Edge is now my preferred browser as Chrome has become resource intensive. In the last 8 months i never had to use Chrome because something did not work, and that is a huge difference compared with about a year ago.
Have been using Chrome from the moment IE8 was released.

Microblocks. Build with logic.
 
bigfix
Senior Member

Joined: 20/02/2014
Location: Austria
Posts: 107
Posted: 09:28am 12 Sep 2017
Copy link to clipboard 
Print this post

There are enough W10 threads already - but just to explain...

You are right about prehistoric - but this W10 is not for real use
I bought a Microsoft Surface Pro to play with W10 & Touchscreen

Actually I wanted to use it partly with a Android Emulator for the rich app store
When it works - it is faster than a real Android tablet - but it randomly works...
The emulator runs much worse with new W10 Versions - so I disabled the updates

I have many Laptops and PCs and they run mostly W7 with all updates
My work Laptop is a QCore i7 with 20Gig memory and 3 SSDs - Chrome runs well

I got sick and tired to chase down, what W10 decided to change the night before
i.e. I got random disk errors by a new USB3 driver, lost my USB connected screen a few weeks later and had a hard time getting it back.
A friend of mine runs a CAD company and W10 killed his main HP workstation by disabling KB & Mouse USB.
You cannot roll back without mouse & keyboard
This costed him many hours of lost time and finally required a reimage...

W10 will be the final reason for me to look into Linux when W7 is deadEdited by bigfix 2017-09-13
 
CaptainBoing

Guru

Joined: 07/09/2016
Location: United Kingdom
Posts: 1731
Posted: 10:22am 12 Sep 2017
Copy link to clipboard 
Print this post

  bigfix said   ...
W10 will be the final reason for me to look into Linux when W7 is dead


+1
 
bigfix
Senior Member

Joined: 20/02/2014
Location: Austria
Posts: 107
Posted: 10:26am 15 Sep 2017
Copy link to clipboard 
Print this post

1.1.2 is here !
no further excuses not try it

New more interactive Demo at
ESPTerm Demo









Wifi Bug is fixed

Frontend got completely rewritten - works better across browsers
Chrome still looks best and also has optional fullscreen mode

Uses HTML5 now, faster rendering
256 Colors, Cursor styles

Many more config options in expert settings

For all details see release notes


ESPTerm Latest release


Best experience with MicroMite is with 230400 Baud, black foregnd, white backgnd
try to stick with one active window, three windows work, but startup is slow

Please use the Mailing Group for feedback and requests to the developers
Mailing List
Edited by bigfix 2017-09-16
 
bigfix
Senior Member

Joined: 20/02/2014
Location: Austria
Posts: 107
Posted: 10:36am 15 Sep 2017
Copy link to clipboard 
Print this post

Useful Code Snippet from Rob to test Mouse functionality:

Const ESC=Chr$(27)

Print ESC "[?9h"; ' enable mouse reporting
Print ESC "[?1015h"; ' select URXVT encoding (if available)
Print ESC "[?1006h"; ' select SGR encoding (if available)

Do
Do: A$=Inkey$: Loop Until A$<>""
If A$=Chr$(27) Then Print
If Asc(A$)<32 Then Print "<" Str$(Asc(A$)) ">"; Else Print A$;
Loop Until A$=Chr$(120)

Print ESC "[?1006l"; ' disable SGR encoding
Print ESC "[?1015l"; ' disable URXVT encoding
Print ESC "[?9l"; ' disable mouse reporting


it makes use of alternative encoding for the mouse location reporting - SGR (1006) is the preferred reporting method, URXVT (1015) is ok but not great, while the default encoding can not encode beyond column 95 if using 7-bit characters.

To exit type "x"

Mousebuttons are encoded in first Value - even the wheel works
Scren shows left button clicks in corners & center



 
bigfix
Senior Member

Joined: 20/02/2014
Location: Austria
Posts: 107
Posted: 11:21pm 22 Oct 2017
Copy link to clipboard 
Print this post

Ondrej, the main author, is back to University, writing his Master
But he implemented some great features to ESPTerm before
I was running some simple tests for a week and it looks very stable to me


ESPTerm Releases


This release was tested with ESP-01, ESP-01S and ESP-12.
It's known to work best in Chrome (Chromium, Brave, Opera), Firefox and Safari.
Internet Explorer is not supported.
MS Edge reportedly works to some degree, but is very slow.



Some 2.2. Features:

Image backgrounds - allow a DYNAMIC background behind Text Forground






Image background can be configured in the Terminal Settings form, or sent using an OSC command:
[Escape] = Chr$27
[Bell] = Chr$7



OSC 70 ; url ST -
eg. [Escape]]70;https://goo.gl/sh4dny[Bell]

is the damselfly from the above screenshot (source: Wikipedia).

The URL is resolved by user's browser, so HTTPS or even FTP can be used.
Animated gifs will work too.
Note: don't use this if you expect users to connect to the built-in AP,
they won't be able to access the image.

This feature is useful for: logos, process diagrams, pre-rendered UI elements, etc.

The URL can be max 100 characters long.
The image will be centered and scaled to fit the screen. Note that about 6px on either side of the screen is used for the margins and the screen size, resp. aspect ratio, may slightly vary with the font used.
Don't expect pixel-perfect graphics, if you need that, use any of the Unicode line / box drawing characters.
The image will be visible where the default background color is used - use [Escape][0m or [Escape][49m to reset to the default color.
The background color around and behind the image is your configured default background color
Clear the background URL box or send [Escape]]70;[Bell] if you want no background image.


Debug parser - select in terminal settings.
this parser captures all input and instead of interpreting the sequences, shows them right on the screen.
Green codes are low ASCII control characters, red are corrupt UTF-8 sequences.

Here's an example sending an image file to the UART.





Partial screen updates
This is a big performance boost, the terminal should now redraw faster in most cases.
Implementing this feature was quite challenging, and to make it happen, we had to completely re-work the message format sent through the WebSocket.



Color pickers in the theme form

click inside the fg / bg text field to open a color picker.





Multiple Languages

we now offer a Czech and German build alongside the main English one
all except the Help and About pages (which are full of text) are translated
because this changes the embedded html pages, at the moment the language cannot be switched other than by flashing a different binary
New download format

each zip package with the localized 0x00000 and 0x40000 bins also includes a readme, check-sums file, and a flash script.
to verify checksums, run sha256sum -c checksums.txt in the inner folder.
the flash script can be customized using env variables (open it for more info)



Networking commands!

Simple messages for ESPTerm-to-ESPTerm messaging
Command to request an external server and get back the response headers and/or body
API endpoint that can be used for remote control from PC (/api/v1/msg, GET or POST)
For more info, please refer to the build-in help page, it explains it in great detail.

This can run intermixed with terminal traffic on the same COM port
It is done with private messages, which are usually ignored by terminals


[Escape] = Chr$27
[Bell] = Chr$7
^ is the plain "Uparrow" Ascii Symbol - not "Control"

Device-to-device Messaging

To send a message to another ESPTerm module or IP Node, use: [Escape]^M;DestIP;message[Bell].

This command sends a POST request to http://<DestIP>/api/v1/msg.
The IP address may be appended by a port, if needed (eg. :8080).
In addition to POST, a GET request can also be used.
In that case, any GET arguments (/api/v1/msg?arguments) will be used instead of the request body.
This is intended for external access when sending POST requests is not convenient.

Each ESPTerm listens for such requests and relays them to UART:
[Escape]^m;SrcIP;L=length;message[Bell],
with length being the byte length of message, as ASCII.

Notice a pattern with the first letter:
capital is always a command, lower case a response.
This is followed with the HTTP commands and any networking commands added in the future.

Example: Node 192.168.0.10 sends a message to 192.168.0.19:
[Escape]^M;192.168.0.19;Hello[Bell].
Node 192.168.0.19 receives [Escape]^m;192.168.0.10;L=5;Hello[Bell] on the UART.

Note that the IP address in the reception message is that of the first node,
thus it can be used to send a message back.

External HTTP requests

To request an external server, use [Escape]^H;method;options;url\nbody[Bell].

method - can be any usual HTTP verb, such as GET, POST, PUT, HEAD.
options - is a comma-separated list of flags and parameters:
H - get response headers
B - get response body
X - ignore the response, return nothing

N=nonce - a custom string that will be added in the options field of the response message.

Use this to keep track of which request a response belongs to.

T=ms - request timeout (default 5000 ms), in milliseconds
L=bytes - limit response length (default 0 = don't limit).
Applies to the head, body, or both combined, depending on the H and B flags
l=bytes - limit the response buffer size (default 5000 B).
This can reduce RAM usage, however it shouldn't be set too small,
as this buffer is used for both headers and the response body.
url - full request URL, including http://. Port may be specified if different from :80,
and GET arguments may be appended to the URL if needed.
body - optional, separated from url by a single line feed character (\n).
This can be used for POST and PUT requests.
Note: the command may be truncated to the maximum total length of 256 characters if too long.
The response has the following format: [Escape]^h;status;options;response[Bell]

status - a HTTP status code, eg. 200 is OK, 404 Not found.
options - similar to those in the request, here describing the response data.

This field can contain comma-separated B, H and L=bytes and N=nonce.
response - the response, as requested.
If both headers and body are received, they will be separated by an empty line
(i.e. \r\n\r\n). Response can be up to several kilobytes long, depending on the L= and l= options.

Example: [Escape]^H;GET;B;http://wtfismyip.com/text[Bell]
- get the body of a web page (wtfismyip.com is a service that sends back your IP address).
A response could be [Escape]^h;200;B,L=11;80.70.60.50[Bell].


 
bigfix
Senior Member

Joined: 20/02/2014
Location: Austria
Posts: 107
Posted: 10:02pm 23 Oct 2017
Copy link to clipboard 
Print this post

A message from Ondrej just came in on the mailing list:


Hello!
A quick update about the current development.
I don't have as much time as before,
so it's going a bit slower, but has not stopped!


Here's a line-up of the features that will, hopefully, be in the next release, 2.3:


- Settings export and import with INI files

this feature is already implemented and working, there will be a testing build soon.
This essentially lets you clone or back-up a well configured module, all in readable,
easy to edit text files that can be archived and exchanged.
An example INI file exported from my testing module:
https://gist.github.com/MightyPork/a7bf8c7b51af6bd90db0acbfbce1f970


- Support for access via port forwarding / https tunnel
(we have some bugs in the front-end code that prevent this from working)



- Configurable button count
a command and setting to lower the number of buttons


- Configurable button colors
again, a command and setting


Let me know if there's any features or fixes you want added (reply here,
or post it on GitHub).
ESPTerm GitHub
ESPTerm Mailinglist


Any small improvements can still be squeezed into this release,
which should be ready in a couple weeks at most.


Cheers
Ondrej
 
panky

Guru

Joined: 02/10/2012
Location: Australia
Posts: 923
Posted: 02:48pm 24 Oct 2017
Copy link to clipboard 
Print this post

Guenter,

A very interesting project and congratulations to Ondrej for his work.

It is not completely clear to me how the Micromite and the esp interact and where the web fits into this - could you explain a little more on how it all ties together please?

A simple diagram might also assist?

Thanks,
Doug.

... almost all of the Maximites, the MicromMites, the MM Extremes, the ArmMites, the PicoMite and loving it!
 
bigfix
Senior Member

Joined: 20/02/2014
Location: Austria
Posts: 107
Posted: 10:38pm 24 Oct 2017
Copy link to clipboard 
Print this post

Doug,
it is really very simple

The ESPTerm acts basically like a VT102 without a physical screen/keyboard

When you flash the code into a ESP and powercycle it, then you will see a new WLAN with the SSID TERM-xxxxx

You connect to this WLAN and get an IP address from the built in DHCP

Then you type in 192.168.4.1 in the Web-Browser (best use Chrome)

If everything is working, you should see a small terminalwindow and a few buttons
You have an integrated Help page, which should explain everything

Then you click on WiFi & Network and configure your preferences
Recommend to join existing network with a fixed IP and disable integrated accesspoint
then powercycle and join the normal network and type the assigned IP address

Now config Terminal to 80x24, dark foreground, light background, capture F5,11,12
- this makes it working smoothly with MMbasic edit

Config Baudrate - recommend 230kbit (same on MMite !!)

As a test you can short the ESP xmit/rcv wires and type some characters
- this should be echoed to the terminalwindow in the browser

Now you should be able to connect the rcv/xmt wires to a MMite (crossed)
and work just like your usual terminal emulation
You can run the MMBasic editor etc...

If you open another window on another PC/Tablet you will see the same screen synced in realtime - Input will be mixed from both
Do not use more than 2 open windows for best performance

If you send the charactersequences from the help page you can steer many things like title line, screen size ...

you can also insert a background in a dynamic fashion
i.e.
Chr$27 + ]70;https://goo.gl/sh4dny = Chr$7
should display a damselfly from the web

The URL string can be max 100 Char long - be careful with naming

So for instance you can create a Weatherstation background with all the bells and whistles on Powerpoint or Photoshop
Then you create i.e. 8 Versions of this, with different wind direction pointers
Store this at your local NAS - or you could trust some internet hoster

Now you dynamically select the right background based on wind direction and send the right ascii sequences to overlay the text fields on the background

This approach has limited scaleabilty if you have more than one variable graphic
i.e. wind direction & weather status gives you 8x4 = 32 pictures

Together with Touch/Mouse enabled (Discussed earlier here) you can create complete GUIs mixing static background with colored text buttonfield overlay
You can create professional looking tabbed menu pages

Benefit
access to MMbasic device from any normal Web browser without special APPs or plugins
dirt cheap user interface ...


What is the tradeoff ?

Currently only WiFi - slim chances of getting wired connectivity with ESP32

No real fonts/sizes in the text foreground - but all attributes/colors

Not integrated in MMbasic with simple commands
You need to build character sequences yourself or create functions

Input parser required, IF you want to use more than classic terminal
Mouse/Touch
KB Input
Per2Peer Networking
WEB Api Networking
can all work concurrently over the same COM port
MMbasic needs to demultiplex those streams

just play with it, use help, then you will understand

cheers
Guenter








Edited by bigfix 2017-10-26
 
Quazee137

Guru

Joined: 07/08/2016
Location: United States
Posts: 350
Posted: 12:39am 26 Oct 2017
Copy link to clipboard 
Print this post


Still waiting for my two seems to be a very slow boat when I was marked airmail.

Firefox fonts


Thanks for getting to know Ondrej and getting this working for us.

Funny thing Ondrej wrote the minecraft editor my Granddaughter uses.
 
bigfix
Senior Member

Joined: 20/02/2014
Location: Austria
Posts: 107
Posted: 07:54pm 25 Sep 2021
Copy link to clipboard 
Print this post

Have a look at
ESPterm releases

there were quite a few useful aditions after my last post in 2017...
 
Print this page


To reply to this topic, you need to log in.

© JAQ Software 2022