Home  |  Contents 
Microcontroller and PC projects
  Forum Index : Microcontroller and PC projects         Section
Subject Topic: Newie got a few questions Post ReplyPost New Topic
<< Prev Page of 33 Next >>
Author
Message << Prev Topic | Next Topic >>
TassyJim
Guru
Guru


Joined: 07 August 2011
Location: Australia
Online Status: Offline
Posts: 2277
Posted: 29 August 2016 at 8:38am | IP Logged Quote TassyJim

If the data is being sent in one burst,
1. read the data
2. add it onto the end of a bugger string
3. wait a brief while
4. go back and get the next chunk and add it onto the end of your buffer string.
If there is no data available after a suitable length wait, you can assume that the sending has finished and you have all the data to play with.

Another option.
You know the maximum amount of data that gets send in one event so you can work out the time it takes to send (depends on the baud rate and number of characters.
When you get the interrupt signalling the data has started to arrive, wait the maximum time before retrieving the data.
That way you know you have got it all in one chunk.

If you know the number of fields arriving, you know the number of commas so you can count them.

If you have control over the sending, add a CR to the end so you know you have it all.

And there are lots of other ways to do the task. AS long as the sending system and the receiving system know what the other is doing.

Jim

__________________
It all started with the ZX81....
VK7JH
http://www.c-com.com.au/MMedit.htm


Back to Top View TassyJim's Profile Search for other posts by TassyJim Visit TassyJim's Homepage
 
WhiteWizzard
Guru
Guru


Joined: 05 April 2013
Location: United Kingdom
Online Status: Offline
Posts: 2436
Posted: 29 August 2016 at 8:56am | IP Logged Quote WhiteWizzard

From the data samples you send, the 'word' is always in speech marks, then there is a colon, then there is the 'data'. Any more 'chunks' of information are separated by a comma; but they follow the same pattern above.

Note the 'data' can be numeric (not in speech marks), or wording (in speech marks).

So an example from your data:

"id":800,"main":"Clear","description":"clear sky"

Are you trying to read the data 'between' commas (i.e. "id":800) or are you trying to read the 'word headings' (i.e. ID) so you can then 'extract' the data (800)?


Edited by WhiteWizzard on 29 August 2016 at 8:58am


__________________
For everything MicroMite visit MicroMite.org

Direct Email: WhiteWizzard@MicroMite.org
Back to Top View WhiteWizzard's Profile Search for other posts by WhiteWizzard Visit WhiteWizzard's Homepage
 
lew247
Guru
Guru
Avatar

Joined: 23 December 2015
Location: United Kingdom
Online Status: Offline
Posts: 790
Posted: 29 August 2016 at 9:03am | IP Logged Quote lew247

Basically I'm going to get the ESP to send the data as follows
"Start", data1,data2, data3 and so on then "end"
That way there will be a start and an end to pick out valid data
EACH item will be seperated by a comma

what I need to do is check there is a start and end being received by the micromite
THEN once there is valid data in the Serial buffer be able to pick out sections of data
for example I might want to PRINT data1 but I might want to do something different Data3
Hope that made sense

Edited by lew247 on 29 August 2016 at 9:04am
Back to Top View lew247's Profile Search for other posts by lew247
 
matherp
Guru
Guru


Joined: 11 December 2012
Location: United Kingdom
Online Status: Offline
Posts: 1851
Posted: 29 August 2016 at 9:09am | IP Logged Quote matherp

The data is in JSON format, see wikipedia

I can't find a full JSON parser written in Basic although MikeO has done some great work on this
Back to Top View matherp's Profile Search for other posts by matherp
 
JohnS
Guru
Guru


Joined: 18 November 2011
Location: United Kingdom
Online Status: Offline
Posts: 1562
Posted: 29 August 2016 at 9:14am | IP Logged Quote JohnS

I'd work on the basis that you need to build up a string char by char from the serial buffer (LOC will tell you if there's at least one) because you have functions that can work on a string.

Then you need to allow for errors such as inserted/corrupted chars.

So you need to allow for things such as a start but no end, two starts etc.

If each item has a trailing comma you can separate the items using that (INSTR can find a comma, for example). If it's something else then use that.

Then you need to allow for each item possibly being corrupted.

And so on. Just think logically. (You can't program anything robust otherwise.)

It might be simpler to write the code if you don't care if it crashes / hangs / is unreliable from time to time. The code itself may be about the same length if you make it robust but may well need harder thought...

John

Edited by JohnS on 29 August 2016 at 9:23am
Back to Top View JohnS's Profile Search for other posts by JohnS
 
jman
Guru
Guru
Avatar

Joined: 12 June 2011
Location: New Zealand
Online Status: Offline
Posts: 662
Posted: 29 August 2016 at 9:43am | IP Logged Quote jman

Hi
Looks like you are trying parse some weather information.
Not sure if this helps but they way I do it is use a web server running PHP
(Most Web Servers can do this)
This downloads and does a parse on the json and then sends the results in nice
linefeed delimited format to the MM. As I know none of the lines are more than
255 characters and I know how many lines to expect reading the data from the com port via the ESP8266 is easy


Do
 Line Input #1, C$
 If echo Then Print c$
Loop While C$ <> "START"

Print "Start found ";C$

Do
 I=I+1
 Line Input #1, WData$(I)
 Print WData$(I)
Loop While WData$(I) <> "END"
 
Print "End Found ";WData$(I)

End Sub


Below is the output from the PHP script of course you can customize
it to suit your needs and taste


START
Mon
Time 21:40:06 29-08-2016
Temperature 08
Humidity 76
Pressure 1008
Mon
29-08-2016
Max : 19
Min : 07
mostlycloudy
Sunrise : 07:01
Sunset  : 17:59
Wind SPD : 4
Mostly cloudy. Low 7C.
Tue
30-08-2016
Max : 14
Min : 03
partlycloudy
Sunrise : 06:59
Sunset  : 18:01
Wind SPD : 13
Some clouds in the morning will give way to mainly sunny skies for the afternoon. High 14C. Winds ESE at 10 to 15 km/h.
Wed
31-08-2016
Max : 13
Min : 02
clear
Sunrise : 06:58
Sunset  : 18:02
Wind SPD : 23
Sunny. High 13C. Winds ENE at 15 to 30 km/h.
Thu
01-09-2016
Max : 16
Min : 07
mostlycloudy
Sunrise : 06:56
Sunset  : 18:03
Wind SPD : 23
Partly cloudy. High 16C. Winds NNW at 15 to 30 km/h.
Fri
02-09-2016
Max : 19
Min : 06
clear
Sunrise : 06:54
Sunset  : 18:04
Wind SPD : 19
Sunshine. High 19C. Winds NNW at 15 to 25 km/h.
Sat
03-09-2016
Max : 13
Min : 06
partlycloudy
Sunrise : 06:52
Sunset  : 18:05
Wind SPD : 14
Partly cloudy. High 14C. Winds SE at 10 to 15 km/h.
Christchurch
END



To use the PHP use the following format
http://your-url-here/test.php?Key=xxxxxx&Location=NZCH
This uses https://www.wunderground.com/ to obtain the weather data you will
need to get an API key from them (There are free).

2016-08-29_093449_Weather.zip

Please note I am no PHP expert so the PHP is not very pretty but it does work
Hope this helps

Regards
Jman

Back to Top View jman's Profile Search for other posts by jman
 
WhiteWizzard
Guru
Guru


Joined: 05 April 2013
Location: United Kingdom
Online Status: Offline
Posts: 2436
Posted: 29 August 2016 at 9:57am | IP Logged Quote WhiteWizzard

lew247 wrote:
. . . be able to pick out sections of data
for example I might want to PRINT data1 but I might want to do something different Data3
Hope that made sense


If data1 always reprints the same data item, then this will be fine, but what if data1 is a different data item? This is why I should think the word (header, call it whatever) is relevant so that you know what the subsequent data represent.

Anyway, if you just want to extract the 'characters' between commas then here is a little demo program for you. I am simply looking for the first comma in a string (with INSTR), then looking if there is another comma, then if so, print the characters between the commas with the MID$ function. Keep going until no more commas, in which case just finally print out the characters after the last comma.

DIM a$ = "Start,123,456,hello,test,1a2s3,,test2, end"    'populate our test string
PRINT a$                   'display our test string
p1 = INSTR(1,a$,",")       'look for position of first comma 
ww:                        'a label which we will jump back to continually until we see no more commas
IF p1<> 0 AND p1 <> LEN(a$) THEN  'check there is a comma and its position isn't at the end of the string
   p2 = INSTR(p1+1,a$,",")        'look for next comma AFTER current comma (i.e. from p1+1)
   IF p2 THEN                     'if we found another comma then now extract data between commas....
      PRINT MID$(a$,p1+1,p2-p1-1) 'for now just print data
                                  'Want data between commas hence start at p1+1, and for number of characters up to second comma i.e. p2-p1-1
      p1=p2                       'so we can loop through string, set new p1 as p2 and look for next comma (into p2 if comma exists)
      GOTO ww                     'go loop through until no more commas
   ELSE
      PRINT MID$(a$,p1+1,LEN(a$)-p1)  'no more commas so print remaining characters
   ENDIF
ENDIF   


Edited by WhiteWizzard on 29 August 2016 at 10:02am


__________________
For everything MicroMite visit MicroMite.org

Direct Email: WhiteWizzard@MicroMite.org
Back to Top View WhiteWizzard's Profile Search for other posts by WhiteWizzard Visit WhiteWizzard's Homepage
 
lew247
Guru
Guru
Avatar

Joined: 23 December 2015
Location: United Kingdom
Online Status: Offline
Posts: 790
Posted: 29 August 2016 at 10:09am | IP Logged Quote lew247

WW that is what I needed thanks, I can work from that to get mine to do what I want.
Thanks everyone else for the help, yes the ESP gets JSON and iv'e already got it parsing only the actual information I need sent to the Micromite.
Back to Top View lew247's Profile Search for other posts by lew247
 
MikeO
Senior Member
Senior Member


Joined: 11 September 2011
Location: Australia
Online Status: Offline
Posts: 138
Posted: 29 August 2016 at 11:09am | IP Logged Quote MikeO

Hi Lew, here is a function I wrote some time ago to do exactly what you are trying to do.
I use this function very frequently and I have found it totaly reliable.


'Parse$, function to return a field from a delimited string
'Usage Parse$(Delimited String, Number of Field in string,[optional delimiter])
'Delimiter defaults to comma unless specified
'eg r$=Parse$("Bat,Ball,Cricket,Pitch",3) will return "Cricket" in r$

Function Parse$(s$,FieldNumber,d$)
Local String stringArg$
Local Integer intOldY,intY,intX
if d$="" then
  d$=","
endif
StringArg$ = S$ + d$
intOldY = 1:intX=0:intY=0
Parse$ = ""
do While intY < Len(StringArg$) And intX < FieldNumber
  intY = Instr(intOldY, StringArg$, d$)
  intX = intX + 1
  If intX = FieldNumber Then
    parse$ = Mid$(StringArg$, intOldY, intY - intOldY)
  Endif
  intOldY = intY + 1
loop
'print s$; intx; inty; intoldy
'print fieldnumber;" ";parse$
End Function


__________________
http://www.codenquilts.com.au
Back to Top View MikeO's Profile Search for other posts by MikeO Visit MikeO's Homepage
 
lew247
Guru
Guru
Avatar

Joined: 23 December 2015
Location: United Kingdom
Online Status: Offline
Posts: 790
Posted: 30 August 2016 at 12:22pm | IP Logged Quote lew247

MikeO wrote:
Hi Lew, here is a function I wrote some time ago to do exactly what you are trying to do.
I use this function very frequently and I have found it totaly reliable.


Thanks Mike, it will be extremely handy and I'm positive it will do what I want,
BUT
Unfortunately I cannot understand a lot of it

I've added comments to the code with the bits I do understand and dont.

IF you don't mind, could you explain what each bit does please?
This will help me immensely and also anyone else who isn't good with this stuff.

Lewis


'Parse$, function to return a field from a delimited string
'Usage Parse$(Delimited String, Number of Field in string,[optional delimiter])
'Delimiter defaults to comma unless specified
'eg r$=Parse$("Bat,Ball,Cricket,Pitch",3) will return "Cricket" in r$  'DO YOU HAVE TO HAVE Bat, Ball, Cricket and Pitch there? 

Function Parse$(s$,FieldNumber,d$)    'IS this the function name? not sure what s$, FieldNumber,d$ means??
Local String stringArg$               'Local saying stringAtg$ is a string? wouldnt Local stringArg$  do the same??
Local Integer intOldY,intY,intX       'Local saying intOldY, intY, intx are integers?
if d$="" then                         'If d$ is empty then d$ = "," ???
  d$=","
endif
StringArg$ = S$ + d$                  'Not sure what this does?
intOldY = 1:intX=0:intY=0             ' Or this one?
Parse$ = ""                            Parse$ is the function name? not sure why it = "" ? 
do While intY < Len(StringArg$) And intX < FieldNumber    'not sure that this means
  intY = Instr(intOldY, StringArg$, d$)                   'or this
  intX = intX + 1                                         'adding 1 to intX? or something else?
  If intX = FieldNumber Then                              ' Not sure what this is? 
    parse$ = Mid$(StringArg$, intOldY, intY - intOldY)    ' umm I must seem really stupid at this point
  Endif
  intOldY = intY + 1                                      'intOldY = inty+1 whatever intY was it's now plus 1 and intOldy?
loop                                                      'Loop 
'print s$; intx; inty; intoldy                            'PRINT these values but not sure what they actually are
'print fieldnumber;" ";parse$                             'Same here
End Function



IF it helps anyone, this is the information coming INTO the serial port of the Micromite

Quote:
Today,broken clouds,04d,19.82,1024,5.1,200,77,Day2,Wed Aug 31 12:00:00 2016,light rain,10d,13.3,19.51,1024,70,5.56,200,Day3,Thu Sep 01 12:00:00 2016,few clouds,02d,10.54,19.69,1024,78,3.67,200,Day4,Fri Sep 02 12:00:00 2016,light rain,10d,12.55,17.38,1024,97,6.39,200,Day5,Sat Sep 03 12:00:00 2016,clear sky,01d,13.78,25.76,1024,0,5.52,200,Day6,Sun Sep 04 12:00:00 2016,clear sky,01d,13.42,26.85,1024,0,2.47,200

'ITEM NUMBERS - not sure if they should start at 0 or 1

1 today
2 desc today
3 icon today
4 temp today
5 pressure
6 wind speed
7 direction
8 humidity
9 day 2
10 date 2 first 3 digits on the left is day
11 desc
12 icon
13 temp min
14 temp max
15 press
16 humid
17 speed
18 dir
19 date 3 first 3 digits on the left is day
20 desc
21 icon
22 temp min
23 temp max
24 press
25 humid
26 speed
27 dir
28 date 4 first 3 digits on the left is day
29 desc
30 icon
31 temp min
32 temp max
33 press
34 humid
35 speed
36 dir
37 date 5 first 3 digits on the left is day
38 desc
39 icon
40 temp min
41 temp max
42 press
43 humid
44 speed
45 dir
46 date 6 first 3 digits on the left is day
47 desc
48 icon
49 temp min
50 temp max
51 press
52 humid
53 speed
54 dir


Basicaly I need to pull out some, or all of the above item numbers to do things with in the main program

Edited by lew247 on 30 August 2016 at 12:25pm
Back to Top View lew247's Profile Search for other posts by lew247
 
MicroBlocks
Guru
Guru
Avatar

Joined: 12 May 2012
Location: Thailand
Online Status: Offline
Posts: 2034
Posted: 30 August 2016 at 4:15pm | IP Logged Quote MicroBlocks

You can use the same function as i posted in the LCD topic you posted.
It is then very easy to use the values.

The one MikeO posted also works but is a bit inefficient when you need many values as the data has to be parsed every single time.

This code will parse the string in a single pass and stores each value in an array

'Required Global variable for GetFieldArray function 
  'Dimension this variable so that it can hold all values 
  Dim GetFieldArray$(55) 

Function GetFieldArray( Record$, Delimiter$, KeepQuotes ) 
  Local Index, Char, InQuote, Count 
  InQuote = 0 
  Count = 0 
  If Delimiter$ = "" Then Delimiter$ = "," 
  For Index = 1 To Len(Record$) 
    Char = Asc(Mid$(Record$, Index, 1)) 
    If Char = 34 Then InQuote = Not InQuote 
    If Not InQuote And Instr(Delimiter$, Chr$(char)) >= 1  Then 
      Count = Count + 1 
    Else 
      If Char <> 34 Or KeepQuotes Then 
        GetFieldArray$(Count) = GetFieldArray$(Count) + Chr$(char) 
      EndIf 
    EndIf 
  Next 
  GetFieldArray = Count + 1 
End Function 


You then use it like this (Here i used a space as a delimiter):


  MyString$ = "Hello The Back Shed users."
  N = GetFieldArray(YourStringHere$, " ")

  for L = 0 to N - 1
     print GetFieldArray$(L)
  next

That will give the result:

Hello
The
Back
Shed
Users.


In you example above you would use:

   N = GetFieldArray(MyData$)

   'You can check N if the number of fields is what you expect
  
   'You can use any value directly by its index
   print GetFieldArray$(25)
   print GetFieldArray$(41)
   print GetFieldArray$(11)
...





Edited by MicroBlocks on 30 August 2016 at 4:17pm


__________________
Microblocks. Build with logic.
Back to Top View MicroBlocks's Profile Search for other posts by MicroBlocks
 
lew247
Guru
Guru
Avatar

Joined: 23 December 2015
Location: United Kingdom
Online Status: Offline
Posts: 790
Posted: 30 August 2016 at 4:48pm | IP Logged Quote lew247

Thanks Jean
I still can't get it working, at the moment the error is
Quote:
[85] Function GetFieldArray( Record$, Delimiter$, KeepQuotes )
Error: Variable type not specified


This is how I implemented your function, not sure if I did it correct or not

IF I try and include Dim GetFieldArray$(55) it comes up with this error
Error: A sub/fun has the same name: GETFIELDARRAY

DIM MYSTRING as string
OPEN "COM3:9200" AS #3                                 'Open port for ESP8266
 IF LOC(#3) > 1 THEN SERIAL3
  END IF  

Function GetFieldArray( Record$, Delimiter$, KeepQuotes )  
  Local Index, Char, InQuote, Count  
  InQuote = 0  
  Count = 0  
  If Delimiter$ = "" Then Delimiter$ = ","  
  For Index = 1 To Len(Record$)  
    Char = Asc(Mid$(Record$, Index, 1))  
    If Char = 34 Then InQuote = Not InQuote  
    If Not InQuote And Instr(Delimiter$, Chr$(char)) >= 1  Then  
      Count = Count + 1  
    Else  
      If Char <> 34 Or KeepQuotes Then  
        GetFieldArray$(Count) = GetFieldArray$(Count) + Chr$(char)  
      EndIf  
    EndIf  
  Next  
  GetFieldArray = Count + 1  
End Function  


SUB SERIAL3 
Local F$ 'Define D$ only for use by this subroutine(ignores any D$ in main code)
LOCAL N as float
F$ = Input$(100, #3)  'Suck everything in buffer out  
MyString$ = F$
N = GetFieldArray(MyString$, ",")
print GetFieldArray$(25) 
   print GetFieldArray$(41) 
   print GetFieldArray$(11) 
End Sub
Back to Top View lew247's Profile Search for other posts by lew247
 


<< Prev Page of 33 Next >>
In the news...
 
Post ReplyPost New Topic
Printable version Printable version
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot delete your posts in this forum
You cannot edit your posts in this forum
You cannot create polls in this forum
You cannot vote in polls in this forum

Powered by Web Wiz Forums version 7.8
Copyright ©2001-2004 Web Wiz Guide

This page was generated in 0.1094 seconds.
Privacy Policy     Process times : 0, 0, 0, 0.11