Home
JAQForum Ver 20.06
Log In or Join  
Active Topics
Local Time 14:48 19 Apr 2024 Privacy Policy
Jump to

Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me.

Forum Index : Microcontroller and PC projects : CMM2 + ESP32 FTP server

Author Message
TassyJim

Guru

Joined: 07/08/2011
Location: Australia
Posts: 5880
Posted: 07:08am 23 Feb 2021
Copy link to clipboard 
Print this post

This is a working version of my FTP server experiment.
It is missing a lot of commands but  should be useful as is.
I will add more commands as I need/desire them.

I have removed a few of the debug pauses and now have reasonable transfer rates:
STOR
Receiving: Batman1.wav
.......................................................................c

Receive concluded,  3051400 bytes in 78.67 secs, 38786 bytes/second.
USER
PASS
SYST
FEAT
HELP
CWD
TYPE
SIZE
TYPE
PORT
LIST

19 files listed
TYPE
PORT
RETR
sending: Batman1.wav
................................................................................
.....................................................................

send concluded.  3051400 bytes in 153.28 secs, 19907 bytes/second.



From the comments:
 ' Server requires ESP32 running latest AT firmware (V2.1.0.0)
 ' For handshaking, GPIO14 (RTS) needs to be connected to any user defined IO pin (default pin is 36)
 ' If you are using an ESP32-S2 series chip, this will be different.
 ' I included a 1k resistor in series to prevent issues if other software uses the RTS pin as an output.
 ' Don't use pins 37 or 40
 '
 ' Passive mode not supported
 ' tested with Total Commander on Windows and Android.
 ' also tested with gFTP on Linux Mint.
 ' a file "FTPserver.ini" is written to the current folder on program exit.
 ' this can be edited to change default values
 ' you can also change the default values in prefData:
 '
 ' a baudrate of 460800 gives file saving @ 38kB/s and 19.9kB/s sending
 ' a higher baudrate doesn't result in any improvement due to limitations in the ESP32 firmware.
 ' Currently, the ESP8266 cannot be used with standard firmware
 '
 ' if the program fails to exit cleanly, the ini file might not get updated.
 ' If you have changed the baudrate, you may have to manually edit the ini file to regain control

 PRINT "<ESC> = quit , X = kick connection, D = toggle Debug, V = toggle Verbose, H = show status"
 PRINT "B = change baudrate, R = reset ESP"


The zip contains a single file.
The ini file gets created on first run so you could start the program and immediately shut down then edit the FTPserver.ini file

The program assumes that you have a working WiFi connection.
The ESP doesn't save the NTP data so that has to be reconfigured on every start.

There are a few places where I should be able to get some improvement in the sending speeds.

FTPserver.zip

Jim
VK7JH
MMedit   MMBasic Help
 
JimDrew
Newbie

Joined: 07/10/2020
Location: United States
Posts: 39
Posted: 11:08am 23 Feb 2021
Copy link to clipboard 
Print this post

That's neat stuff!  I *love* TotalCommander.  I have been using it for many, many years - long before Microsoft required them to change the name!
 
JimDrew
Newbie

Joined: 07/10/2020
Location: United States
Posts: 39
Posted: 11:37am 23 Feb 2021
Copy link to clipboard 
Print this post

Wow... just looked at your code.  That is a TON of work to support the stock ESP firmware!  If you were to use modem emulation firmware you can easily increase the speed _A_LOT_!

For a modem emulation you would just set the baud rate and connect to the URL and then you are just passing data without any commands required.  So, no doing a AT+CIPSEND command, and then having to wait on the response of a > and then sending your data block - you just send the data and that's it.  Any response from the server will come through as necessary.  I can really see why there is a horrible bottleneck in speed and why you would need flow control going to the ESP using this interface. Kudos to you for making it work at all!  :)
 
TassyJim

Guru

Joined: 07/08/2011
Location: Australia
Posts: 5880
Posted: 06:55am 02 Mar 2021
Copy link to clipboard 
Print this post

I have made some progress.
A few missing commands have been added and file names with spaces in them are now handled correctly.
The program will still crash out if you try and do something like delete a directory that is not empty.

I am getting reliable speeds of just under 70kB/s sending to the CMM2 and 39kB/s sending from the CMM2.

The program also now runs on the ARMite F407 although at slower transfer speeds. I have to do some more tweaking for the F4.

The top transfer speeds only appear with large files. 200k or more.

As an example of real life, I transferred a program that consists of 136 files in 7 directories totaling 9.5MB.
The transfer took 3min 45 secs. I didn't have to create any directories. That all happens complements of Total Commander.

There is still a lot more work to do but the program is very usable as is.


FTPserver.zip

Jim
VK7JH
MMedit   MMBasic Help
 
jirsoft

Guru

Joined: 18/09/2020
Location: Czech Republic
Posts: 532
Posted: 11:22am 02 Mar 2021
Copy link to clipboard 
Print this post

Hello Jim,
may I integrate it (or parts of it) into Napoleon Commander? I'm doing some bigger modifications now (mainly networking), so it could make sense...
Jiri
Napoleon Commander and SimplEd for CMM2 (GitHub),  CMM2.fun
 
TassyJim

Guru

Joined: 07/08/2011
Location: Australia
Posts: 5880
Posted: 08:14pm 02 Mar 2021
Copy link to clipboard 
Print this post

  jirsoft said  Hello Jim,
may I integrate it (or parts of it) into Napoleon Commander? I'm doing some bigger modifications now (mainly networking), so it could make sense...


Use all or as much as you like.

I would be very interested if you can come up with  firmware for the ESP-01 module that works for FTP.

Jim
VK7JH
MMedit   MMBasic Help
 
TassyJim

Guru

Joined: 07/08/2011
Location: Australia
Posts: 5880
Posted: 06:00am 03 Mar 2021
Copy link to clipboard 
Print this post

Another small update (mainly tidy up and check against FTP specs)

  Quote  FTPserver.bas is intended to be run on a Maximite computer to make file transferring easier.
It was developed on a Colour Maximite 2 and has been tested on an ARMite F407.

It uses a subset of the FTP commands and has been tested with Total Commander on Windows and Android as well as gFTP on Linux.
It requires a ESP32 running the standard AT command set firmware. Tested with V2.1.0.0
It requires a serial port connection as well as one handshake line.
For handshaking, GPIO14 (RTS) needs to be connected to any user defined IO pin (default pin is 36)
If you are using an ESP32-S2 series chip, this will be different.
I included a 1k resistor in series to prevent issues if other software uses the RTS pin as an output.
Don't use pins 37 or 40 on the CMM2

I had difficulties getting the ESP8266 running due to problems when the data stream is closed.

The program creates a file FTPserver.ini in the program folders with your preferences.
The easiest way to configure the program to suit your installation is to run the program then quit (using <ESC>). This will create the text file which you can then edit.

By default, the program assumes that your WiFi is already configured. If not, Option 'W' will try and connect using the SSID and password in the ini file.
Once you have the ESP32 connecting on startup, you may wish to change the SSID and password to a generic name to protect your true password from prying eyes.

Current state of commands implemented:

USER any username is accepted and password requested.
PASS any password is accepted
CWD   change working directory
CDUP change up one level
QUIT gracefull exit and go back to waiting for connection
PORT client specifies port for data.(Active mode)
TYPE A (ASCII) and I (Image or binary) are both accepted but are really both binary.
No second paramater accepted
RETR retrieve. send file from the server
STOR store. send file to the server.
RNFR rename from
RNTO rename to. These work as a pair of commands
ABOR Abort file transfer
DELE delete file or folder. No protection against trying to delete a non-empty folder.
RMD delete a folder. No protection against trying to delete a non-empty folder.
MKD Make directory. create new folder.
PWD Print name of current working directory.
LIST List files in current folder. Includes 'fake' permissions and size and date formatted.
Does not accept a path or file name so only current directory available.
SITE SITE CHMOD accepted but faked and success reply sent
NOOP OK responce sent

FEAT returns "SIZE" as the only feature
SIZE returns size of given file.
SYST system. returns "215 UNIX emulated by MMBasic"


PASV Passive mode NOT supported (you can't have two servers enabled)
MODE not supported default is 'stream'
STOU not supported
APPE not supported
ALLO not supported
REST not supported
NLST Not supported
STAT not supported
HELP not supported



FTPserver.zip

Jim
VK7JH
MMedit   MMBasic Help
 
TassyJim

Guru

Joined: 07/08/2011
Location: Australia
Posts: 5880
Posted: 12:15am 08 Mar 2021
Copy link to clipboard 
Print this post

This update uses a few fixes that Peter has done for the F4
I have been spoilt by the speed of the CMM2!


FTPserver.zip

Jim
VK7JH
MMedit   MMBasic Help
 
thwill

Guru

Joined: 16/09/2019
Location: United Kingdom
Posts: 3830
Posted: 09:48am 08 Mar 2021
Copy link to clipboard 
Print this post

Hi Jim,

Is this https://www.ebay.co.uk/itm/ESP32-WROOM-32-38-pins-Development-Board-Dual-Core-2-4-GHz-WLAN-WiFi-Bluetooth/310806001965 the correct ESP32 hardware for this program ? If not then could you please point me at what I should be buying ... even if the one I've specified *might* work then perhaps you should point me at *exactly* the one you are using, it would make troubleshooting easier.

Best wishes,

Tom
Game*Mite, CMM2 Welcome Tape, Creaky old text adventures
 
lizby
Guru

Joined: 17/05/2016
Location: United States
Posts: 3008
Posted: 12:46pm 08 Mar 2021
Copy link to clipboard 
Print this post

Don't know what Jim is using, but my favorite ESP32 is the "D1 Mini" version:
https://www.ebay.co.uk/itm/MINI-Wemos-D1-ESP32-ESP-32-WIFI-Development-Bluetooth-CP2104-Module-ESP8266-KIT/284117671026

It's less breadboard-friendly than the one you linked to, but it's compatible with the ESP8266 D1 Mini, and I have made a breakout PCB for it.

I haven't yet tried the FTP program. Still mostly making do with XMODEM R and the occasional card swap (despite there being other options available).
PicoMite, Armmite F4, SensorKits, MMBasic Hardware, Games, etc. on fruitoftheshed
 
TassyJim

Guru

Joined: 07/08/2011
Location: Australia
Posts: 5880
Posted: 08:33pm 08 Mar 2021
Copy link to clipboard 
Print this post

The only pins I have used are the serial port ones.
I am not using CTS (pin GPIO15) but it is worth having it available for future uses.
The layout of the board does not matter. As long as it has the 4 IO pins available - GPIO 14,15,16,16.
They are the pins that the current firmware uses but I can not guarantee that the next firmware uses the same pins.



The board you are looking at should be fine. They have the required pins broken out (if the photo is to be believed)

The ones I have use this layout:


The GPIOs are in the correct position but the 'purpose' for each pin is not necessarily correct on the diagram.

No point referring to the exact purchase I made, it was an Australian supplier and the item has been 'updated' since my purchase.

Jim
VK7JH
MMedit   MMBasic Help
 
thwill

Guru

Joined: 16/09/2019
Location: United Kingdom
Posts: 3830
Posted: 12:01pm 09 Mar 2021
Copy link to clipboard 
Print this post

Thanks Jim,

Am I also going to need a firmware programmer for the ESP32 of some sort ?

Best wishes,

Tom
Game*Mite, CMM2 Welcome Tape, Creaky old text adventures
 
lizby
Guru

Joined: 17/05/2016
Location: United States
Posts: 3008
Posted: 01:59pm 09 Mar 2021
Copy link to clipboard 
Print this post

  thwill said  Am I also going to need a firmware programmer for the ESP32 of some sort

Modules with USB onboard should not need additional hardware (unlike the ESP-01).

If you need to update the firmware, there are a number of uploaders available, including the one in the zip file in matherp's ESP-01 thread here or from Expressif here .

I'm not sure where the latest ESP32 AT firmware is (if that is needed).

~
Edited 2021-03-10 00:04 by lizby
PicoMite, Armmite F4, SensorKits, MMBasic Hardware, Games, etc. on fruitoftheshed
 
TassyJim

Guru

Joined: 07/08/2011
Location: Australia
Posts: 5880
Posted: 08:35pm 09 Mar 2021
Copy link to clipboard 
Print this post

Latest AT binary (if needed)
https://docs.espressif.com/projects/esp-at/en/latest/AT_Binary_Lists/ESP32_AT_binaries.html

You will need the first one on the page.

espressif also have plenty of documentation available.
It can take a bit to sort out which docs you want.

Jim
VK7JH
MMedit   MMBasic Help
 
Print this page


To reply to this topic, you need to log in.

© JAQ Software 2024