Home
JAQForum Ver 20.06
Log In or Join  
Active Topics
Local Time 00:04 30 Apr 2024 Privacy Policy
Jump to

Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me.

Forum Index : Microcontroller and PC projects : RS232 - Data issues

Author Message
MolsonB
Regular Member

Joined: 25/01/2015
Location: Canada
Posts: 48
Posted: 06:08am 11 Apr 2015
Copy link to clipboard 
Print this post

I can't seem to receive correct RS232 serial data, just garbage. I put a 22KOhm resister and clamping diode on the TX line as per MM Manual (pg. 70). Putting a scope on the line, it brings the levels down to +5.2V ~ -5.8V. The orginal levels are +6.4V ~ -6.2V. I thought following the MM manual would bring the levels closer to 3.3V.

SPECS:
  Quote  
RS-232 @ 57600. Each byte is transferred as a 9-bit byte using 8 data bits, 1 stop bit, and no parity (8N1). When multiple bytes are required for a given parameter (e.g., 16 or 32 bit data values) then the most significant byte shall be transferred first.
Each message shall consist of an eight byte header and single byte footer. In between the message header and footer is the message payload section. The size of the payload section varies and is dependent on the message type.




CODE:

DIM Storage$
DIM MessageLength

OPEN "COM1: 57600, 256, SerialInt, 1, INV" AS #5

'start:
' Print "Waiting for data..."

Storage$ = ""
MessageLength = 20

SETTICK 500, ProcessMessages
Loopything:
' Print Time$
GoTo loopything


ProcessMessages:
Print "Waiting for data..."
A$=Retrieve$()
IF LEN(A$) > 0 then Process(A$)
IRETURN


SerialInt:
DO
a$ = INPUT$(25, #5)
'user function Store to be written, only add to 'Store' when a$ contains data
' print a$
if a$ <> "" then
b = Stores(a$)
End If
LOOP UNTIL a$ =""
IReturn


Sub Process(message$)
print "MESSAGE: " message$
ENd Sub


FUNCTION Retrieve$()
LOCAL Message$
LOCAL SeparatorPosition
'retry:
IF (LEN(Storage$) >= MessageLength) THEN
'At least one possible message in storage
'Find the message separator
'SeperatorPosition = INSTR(Storage$,CHR$(13))
SeperatorPosition = INSTR(Storage$,"$")

' IF (SeperatorPosition <> MessageLength) THEN
'Corrupt message (too short or too long) at the beginning of the Storage
'Correct it by removing the corrupt part
' Storage$ = MID$(Storage$, SeperatorPosition + 1)
' GOTO retry
' ENDIF

'Extract the message
Retrieve$ = MID$(Storage$, 1, MessageLength)
'Remove from storage
Storage$ = MID$(Storage$, MessageLength+1)

ELSE
'Storage contains less then MessageLength characters. No complete message available
Retrieve$ = ""
ENDIF
END FUNCTION



Function Stores(message$)
LOCAL MsgLength
' MsgLength = -1;
MsgLength = LEN(message$)
IF (LEN(Storage$) + MsgLength > 255) THEN
'The error will stop program execution. Good while testing
print Storage$
ERROR "Not enough storage available!"
'In normal mode set MsgLength to -1.
'This will inform the caller that the message is not stored.
MsgLength = -1;
ELSE
Storage$ = Storage$ + message$
ENDIF
Stores = MsgLength
END FUNCTION



OUTPUT:
MESSAGE: ä ÿ ÿ kÿÿÿç~
Waiting for data...
MESSAGE:  € -LD ä ÿ
Waiting for data...
MESSAGE: ÿ 'ÿÿç~ € -LD
Waiting for data...
MESSAGE: ä ÿ ÿ 'ÿÿç~
Waiting for data...
MESSAGE: € -LD ä ÿ ÿ
Waiting for data...
MESSAGE: 'ÿÿÿç~ € -LD
Waiting for data...
MESSAGE: ä ÿ ÿ 'ÿÿÿÿç~
Waiting for data...
MESSAGE: 9€ -LD ä ÿ
Waiting for data...
MESSAGE: ÿ ÿÿç~ € -LD
Waiting for data...
MESSAGE: ä ÿ ÿ 'ÿÿÿç~


What do you guys think? I have the bauds correct, which usually leads to data not displaying correct.
MkII 44pin - v5.0

ColorMax 2 - v4.5
 
BobD

Guru

Joined: 07/12/2011
Location: Australia
Posts: 935
Posted: 06:46am 11 Apr 2015
Copy link to clipboard 
Print this post

If you are referring to a ColorMax 2 like your signature suggests then it seems that the serial ports top out at 19200. However, I couldn't find a manual that matched with serial info on page 70. The other thought was that if the serial pins in use (which pins?) are specified for 3.3 volts then they are probably running far too high at 5.2 volts.

What product are you using and what manual are you referring to?
 
MolsonB
Regular Member

Joined: 25/01/2015
Location: Canada
Posts: 48
Posted: 07:05am 11 Apr 2015
Copy link to clipboard 
Print this post

Sorry, I just got a mkii (44pin, 4.6b version) last night from Micromite.org (highly highly recommend their business) and started to play around with it. I'll update my Sig when I'm back at my desk.

Page 70 from the latest mkii manual shows how to bring rs232 down to TTL logic. I'm not sure if it should be clamping right to 3.3volts, or if the 5volt signal is safe enough.
*EDIT: I was using the scope while the TX line wasn't connected (in fear of blowing the chip with circuit). When I connect the line up, the scope reads +4.5V ~ -1V.

Using com1 (pins 8 & 9 on MKii 44pin), which hardware supports the baud. I have a max 232 chip that I may end up using but hoping the mkii can handle it. I know I'm not handling the serial content correctly right now, I'm just trying to get valid data first.Edited by MolsonB 2015-04-12
MkII 44pin - v5.0

ColorMax 2 - v4.5
 
ajkw
Senior Member

Joined: 29/06/2011
Location: Australia
Posts: 290
Posted: 11:40am 11 Apr 2015
Copy link to clipboard 
Print this post

It seems to be a signalling problem. The hardware hack (for want of another word) may not support 57600, the example in the manual is at 4800 baud.

Can you test it at a slower baud rate?

What does the waveform look like on the CRO? Square or sloping edges?

Edited by ajkw 2015-04-12
 
BobD

Guru

Joined: 07/12/2011
Location: Australia
Posts: 935
Posted: 01:51pm 11 Apr 2015
Copy link to clipboard 
Print this post

If you suspect a bit rate problem and it does look like that, then you can check the bit rate on your oscilloscope. At 57600 speed a bit should be a little over 17 microseconds long. You should see enough single bits in any data stream for easy measurement. If you find a significant variation in the length of the bits then divide that bit length in microseconds into 1,000,000 to get your baud. Sync on the leading edge of a start bit and expand your time base to have one character across the screen. If you have storage then is easy.

edit:
Given you are running at 5 volts then you may have to specify the OC switch on your serial open command and use a pull up resistor to 5 volts on the PIC transmit pin. However, that won't directly affect what you are receiving at the PIC.Edited by BobD 2015-04-12
 
TassyJim

Guru

Joined: 07/08/2011
Location: Australia
Posts: 5905
Posted: 04:55pm 11 Apr 2015
Copy link to clipboard 
Print this post

Are you sure that the data is ASCII?
The specs you have quoted are not clear about that (to me).

Can you get a readable output connecting to a RS232 port on your PC using TeraTerm etc.

If you want to reduce the voltage levels to be closer to 0-3.3V, I would use Schottky diodes as they have a much lower forward voltage drop (and faster acting). I also like to have one going to ground to limit the negative voltage swing.




With the extra diode and using Schottky or other low forward voltage drop diodes the swing should be -0.2 to +3.5V

VK7JH
MMedit   MMBasic Help
 
MolsonB
Regular Member

Joined: 25/01/2015
Location: Canada
Posts: 48
Posted: 05:43pm 11 Apr 2015
Copy link to clipboard 
Print this post

I will admit I haven't done serial communication before...
Good old ASC function to the rescue. The data wasn't ASCII, you nailed it. I've been playing with bluetooth data for to long, where things are nice and friendly to send.

Here's a snip-it right from their PDF file.


My looping output now displays the ascii values.
231
126
0
13
0
127


I'm still a little scared of hooking it right to the MK, so I'll probably ending up putting in a max232 in there. Atleast I know things work.
MkII 44pin - v5.0

ColorMax 2 - v4.5
 
MolsonB
Regular Member

Joined: 25/01/2015
Location: Canada
Posts: 48
Posted: 01:25pm 12 Apr 2015
Copy link to clipboard 
Print this post

I could use your guys help. How do I go about processing the data, comparing, reading, etc. Do I leave it in string format, convert to ascii, hex?

INPUT$ brings in the value as a string.
ASC converts that string to ACSII code so I can actually read it.
The program sends all their values as HEX 0xE7, 0x7E, etc..

When I'm looking for the header key values, do I check for E7, or convert to ascii and check 231 ?

I'm getting tangled up in the basics of serial communication.
MkII 44pin - v5.0

ColorMax 2 - v4.5
 
Grogster

Admin Group

Joined: 31/12/2012
Location: New Zealand
Posts: 9063
Posted: 02:45pm 12 Apr 2015
Copy link to clipboard 
Print this post

Welcome to the wonderful world of serial comms.

As the data is not ASCII, you would probably be best leaving it as a string.
This means you can't visually print the "Characters" as they are not ASCII, but you can still use your storage string idea to hold the data as it comes in.

It would be helpful if the sending module tacked some kind of end-of-message marker to the message such as CR or LF or both. Do you know if your module does that?

IE: Header bytes, then payload bytes, then CR or LF or both as end-of-message marker bytes.
Smoke makes things work. When the smoke gets out, it stops!
 
MolsonB
Regular Member

Joined: 25/01/2015
Location: Canada
Posts: 48
Posted: 02:48pm 23 Apr 2015
Copy link to clipboard 
Print this post

The module does not send an end-of-message marker, just the first 2 hardcoded header bytes. I'm ok with that, I check if there is a complete header (9 bytes and more) and the hardcoded header markers at the start, to process the message.

The last byte is a checksum, and this is what I have.

Document code
For (i=0, checksum=0; i<cnt; i++)
checksum = checksum ^ data; 'XOR all the bytes

checksum = ~checksum 'invert value


MM code

DO
CheckSumCal = CheckSumCal XOR ASC(MID$(Message, x, 1))
x = x + 1
LOOP UNTIL x = Len(Message)
CheckSumCal = CheckSumCal XOR 255 'Invert the summed value

If ASC(CheckSum) = CheckSumCal Then
ValidMessage = Message
Else
ERROR "exit"
End If


To get this straight.
INPUT$ is the only way to get the input from the serial port, and it comes in STIRNG format. Example, the device sends out HEX 0x21 which MM reads the string as "!". If I want to look for 0x21, I have to convert the string to a number (ASC), and then to HEX? Just seems like there is lots of steps. It's being sent as HEX, but gets received as a string, which then have to convert the string to a number, then to a hex value. I want to keep HEX as the document is written using HEX, so it's easy for me to follow along.


Header1 = Hex$(ASC(serialMsg))
If Header1 = Hex$(&h21) Then ....



MkII 44pin - v5.0

ColorMax 2 - v4.5
 
Grogster

Admin Group

Joined: 31/12/2012
Location: New Zealand
Posts: 9063
Posted: 03:42pm 23 Apr 2015
Copy link to clipboard 
Print this post


  MolsonB said  
To get this straight.
INPUT$ is the only way to get the input from the serial port, and it comes in STIRNG format. Example, the device sends out HEX 0x21 which MM reads the string as "!". If I want to look for 0x21, I have to convert the string to a number (ASC), and then to HEX? Just seems like there is lots of steps. It's being sent as HEX, but gets received as a string, which then have to convert the string to a number, then to a hex value. I want to keep HEX as the document is written using HEX, so it's easy for me to follow along.


Yeah, from what I can see, INPUT$ is the only way to get data from the serial port, and if you need numerical data, you have to use ASC. I may be missing something in the manual etc, so I am sure others will let us both know if there is another way.
Smoke makes things work. When the smoke gets out, it stops!
 
MicroBlocks

Guru

Joined: 12/05/2012
Location: Thailand
Posts: 2209
Posted: 05:45pm 23 Apr 2015
Copy link to clipboard 
Print this post

Use HEX notation so you can compare numbers which are in the standard decimal notation with it directly.
What you have is BINARY information. Each byte contains a value. You already have that value with the ASC function.
Just compare it directly with the value and use the notation that is easiest.
The decimal number 33 can be written as &h21 or &O41 and even as &b100001.

samples:


A$ = "A"

if A$ = chr$(65) ....
if A$ = chr$(&H41) ....
if A$ = chr$(&O101) ....

The easiest will be:
if A$ = "A"

A = ASC(A$)
if A = 65 ......
if A = &H41 .....
if A = &O101

the most meaningful might be
if A = ASC("A")


Lots of ways to get the same result.
It all depends in what notation or format your information is. In your case where hexidecimal notation is used stick with the &H.

Edited by TZAdvantage 2015-04-25
Microblocks. Build with logic.
 
MolsonB
Regular Member

Joined: 25/01/2015
Location: Canada
Posts: 48
Posted: 07:30pm 23 Apr 2015
Copy link to clipboard 
Print this post

I think I was getting hung up on comparing what I received on the serial port using HEX$(value), instead of using CHR$(&h21). I'm slowly making head-way.

Trying to figure out how to process the serial Q faster then the data is being sent in. SETTICK 10, just isn't fast enough and the appending string gets more then 255 bytes in it. Not sure if I post a new topic or continue in this one. (Not sure the general forum rules)
MkII 44pin - v5.0

ColorMax 2 - v4.5
 
MicroBlocks

Guru

Joined: 12/05/2012
Location: Thailand
Posts: 2209
Posted: 04:50am 24 Apr 2015
Copy link to clipboard 
Print this post

Create an array to make a circular buffer. Use a head and tail index pointer to know where to store (head) and where to retrieve (tail) data.
Store the received data in the array indexed by the head, then increase the head counter with one and test for end of array. When that happens the head will be 0.
This will be more efficient then using a string and append data to it. That takes considerable more time. Also do the storing of the string in the interrupt routine itself. Saves time calling and returning from a function/subroutine.
You can then read from the same array but use the 'tail' index position. Make sure the tail is not the same as head because that will mean there is nothing in the 'buffer'.
You can do that in the 'loopything' or using a settick. The settick is nicer and allows the 'loopything'to be used for user input or screen updates.
After the read increase the tail and also check for end of array and set it to 0 when end off array is reached.


Microblocks. Build with logic.
 
MolsonB
Regular Member

Joined: 25/01/2015
Location: Canada
Posts: 48
Posted: 07:57am 25 Apr 2015
Copy link to clipboard 
Print this post

Ok I changed things around to use an array buffer, but the MM seems it still can't keep up to the serial data input. Which I don't understand why and what I'm doing wrong.

The longest serial message is 75 bytes, if the length gets past that then the message is corrupt. Since I have to read 1 byte at a time, I limited the buffer array string to 1 length. Basically I'm reading COM1 data from an external serial device, and passing it to Com2 bluetooth to my android tablet.


Dim STRING S1Buffer(512) LENGTH 1, S1Message LENGTH 75, S1MsgPast LENGTH 1
DIM S1Capacity=512, S1Counter=0, S1Head=0, S1Tail=0, S1MsgFlag=0, S1MsgLen=0

OPEN "COM1: 57600, 256, S1_WriteBuf, 1, INV" AS #1
OPEN "COM2: 9600, 256, Serial2_Int, 1" AS #2

SETTICK 2, S1_ReadBuf, 1

Print "Program Started:" TIME$
Print #2, "Program Started:" TIME$

Loopything:
GoTo loopything


S1_WriteBuf:
Local string msg LENGTH 1

msg = INPUT$(1, #1)
IF S1Counter = S1Capacity then
S1Counter = 0
ERROR "Serial 1 Buffer FULL"
End If

S1Buffer(S1Tail) = msg
S1Tail = S1Tail + 1
IF S1Tail > S1Capacity then S1Tail = 0
S1Counter = S1Counter + 1

IReturn



S1_ReadBuf:
local string msg LENGTH 1

If S1Counter > 0 then
msg = S1Buffer(S1Head)
S1Head = S1Head + 1
IF S1Head > S1Capacity THEN S1Head = 0
S1Counter = S1Counter - 1

If S1MsgPast = CHR$(231) AND msg = CHR$(126) THen 'Start of a new message (Header1 0xE7 Header2 0x7E)
S1MsgFlag = 1
S1Message = CHR$(231)
End If
If Len(S1Message) > 74 Then S1Message = "" 'If data is longer then 74, something went wrong and reset
If S1MsgFlag = 1 Then S1Message = S1Message + msg 'Append byte from Buffer to Message string

If Len(S1Message) = 8 Then S1MsgLen = ASC(msg) 'Payload length section

If Len(S1Message) = S1MsgLen+9 Then 'Read the whole message, payload + header

S1_Process(S1Message)
S1MsgFlag = 0
S1Message = ""
ENd If
S1MsgPast = msg

End If

IRETURN



SUB S1_Process(msg as String LENGTH 75)

Local STRING MajorId LENGTH 1, MinorId LENGTH 1, CommandNum LENGTH 1, Source LENGTH 1, Destination LENGTH 1, PayloadData LENGTH 74, CheckSum LENGTH 1
Local x=0, PayloadLength=0, CheckSumCal=0

MajorId = MID$(msg,3,1)
MinorId = MID$(msg,4,1)
CommandNum = MID$(msg,5,1)
Source = MID$(msg,6,1)
Destination = MID$(msg,7,1)
PayloadLength = ASC(MID$(msg,8,1))
PayloadData = MID$(msg,9,PayloadLength)
CheckSum = RIGHT$(msg,1) 'last byte
x = 1
DO
CheckSumCal = CheckSumCal XOR ASC(MID$(msg, x, 1))
x = x + 1
LOOP UNTIL x = Len(msg)
CheckSumCal = CheckSumCal XOR 255 'Invert the summed value

If ASC(CheckSum) <> CheckSumCal Then
PRINT "Error: Corrupt message " CheckSum " = " HEX$(CheckSumCal)
Exit Sub
End If


PRint #2, "VPX_Status:" msg 'SEND to Tablet


End If

ENd Sub


MkII 44pin - v5.0

ColorMax 2 - v4.5
 
MolsonB
Regular Member

Joined: 25/01/2015
Location: Canada
Posts: 48
Posted: 05:24am 26 Apr 2015
Copy link to clipboard 
Print this post

I tried a different approach that is tailored just to this transmission. Within all the data coming fast at me @ 57600 baud that I can't seem to process fast enough, I'm really only interested in 2 messages. They get sent every 1Hz (1 second). I only want the latest message, don't care about the message a few seconds ago still in the buffer Q.

I know this isn't the right way to handle the serial stream, but within the com interrupt I process the data.
1. Flag a new message when the 2 header bytes are detected
2. Check the next 2 bytes which tells what kind of message is coming in (I filter out the ones I don't want to see so they don't get processed)
3. Append to a string.
4. The lengths of the messages vary, have to read the payload length byte first
5. Once the full length is in, save to a specific variable and start again. This way the latest and greatest message gets stored in their own variable.

'****************************** SERIAL 1 ***************************
S1_WriteBuf:

Local string msg LENGTH 1
DO
msg = INPUT$(1, #1)

If S1MsgPast = CHR$(VPX_Header1) AND msg = CHR$(VPX_Header2) THen 'Start of a new message (Header1 0xE7 Header2 0x7E)
S1MsgFlag = 1
S1Message = CHR$(VPX_Header1)
End If

If Len(S1Message) = 3 AND S1MsgPast = CHR$(&h12) Then 'Major - Check what contstant rate message we are about to receive
If (msg = CHR$(&h02) OR msg = CHR$(&h03)) THen S1MsgFlag = 0 'Minor - Do not care about flap & fault
End If

If S1MsgFlag = 1 Then
S1Message = S1Message + msg 'Append byte from Buffer to Message string

If Len(S1Message) = 8 Then S1MsgLen = ASC(msg) 'Payload length section

If Len(S1Message) = S1MsgLen+9 Then 'Read the whole message, header + payload

If INSTR(S1Message,CHR$(&h12) + CHR$(&h00)) > 0 Then VPX_SystemStatus = S1Message
If INSTR(S1Message,CHR$(&h12) + CHR$(&h01)) > 0 Then VPX_DeviceStatus = S1Message

S1MsgFlag = 0
ENd If
ENd If

S1MsgPast = msg
LOOP UNTIL msg=""

IReturn

MkII 44pin - v5.0

ColorMax 2 - v4.5
 
MicroBlocks

Guru

Joined: 12/05/2012
Location: Thailand
Posts: 2209
Posted: 09:44pm 26 Apr 2015
Copy link to clipboard 
Print this post

A message every second is slow. The mcu can keep up with that with no problem.
The baudrate is not the problem as you use a hardware uart with receive and send buffers.

You do need to read all bytes in one go in the interrupt routine.
Reading them one by one is too much overhead.

Try this: *This is not tested just written here as a post
[code]
DIM STRING Buffer(15) LENGTH 255
DIM Buffer.Tail=0, Buffer.Head=0

CONST Buffer.Capacity = 16
CONST Buffer.MaxLength = 255

'Replace 15 with the minimum message length.
OPEN "COM1: 57600, 512, Serial_Receive, 15, INV" AS #1

'Adjust this to be quick enough
'When you see the 'Buffer full!' message then first decrease the interval'
SETTICK 500, ProcessMessages, 1

Print "Program Started:" TIME$

DO:LOOP

SUB Serial_Receive()
DO WHILE LOC(#1) > 0
Buffer.Write INPUT$(Buffer.MaxLength,#1)
LOOP
END SUB

SUB Buffer.Write(Value$)
IF Buffer.Tail = ((Buffer.Head - 1 + Buffer.Capacity) MOD Buffer.Capacity) THEN
Print 'Buffer full'
ENDIF
Buffer(Buffer.Tail) = Value$
Buffer.Tail = (Buffer.Tail + 1) MOD Buffer.Capacity
END SUB

FUNCTION Buffer.Read$()
IF Buffer.Head <> Buffer.Tail THEN
Buffer.Read = Buffer(Buffer.Head);
Buffer.Head = (Buffer.Head + 1) MOD Buffer.Capacity
ENDIF
END FUNCTION

SUB ProcessMessages()
LOCAL Message$
Message$ = Buffer.Read()
DO WHILE LEN(Message$) > 0
'Do your stuff here
'For now just print the length of the string
PRINT LEN(Message$)
Message$ = Buffer.Read()
LOOP
END SUB

[/code]

When running this you would see the lengths of the strings that are placed in the buffer.
Adjust the threshold to the average message length first. This will increase efficiency.
The value is now 15.
Hopefully you will not see a 'Buffer full!' message.
If you do, change the interval 500 to a smaller value. I would decrease it with 50 until you see no more 'Buffer full' happening.
You will then know if the uMite is fast enough to handle the datastream.

Then add your processing of the messages. This processing should not take more time then the used interval!

Edit:Cleaned it up a bit. :)Edited by TZAdvantage 2015-04-28
Microblocks. Build with logic.
 
MolsonB
Regular Member

Joined: 25/01/2015
Location: Canada
Posts: 48
Posted: 05:37pm 27 Apr 2015
Copy link to clipboard 
Print this post

That was the other idea in my head, to read more bytes instead of 1 by 1. Since it's likely the message will be split into the next buffer message, I'll have to store part1 and read part2. I'm taking Wed off, so I'll get to play around with this during the day.

Thanks for another way to look at it.
MkII 44pin - v5.0

ColorMax 2 - v4.5
 
Print this page


To reply to this topic, you need to log in.

© JAQ Software 2024