uM2(+): data logging - cheap fast & easy
| Author | Message | ||||
| matherp Guru Joined: 11/12/2012 Location: United KingdomPosts: 11980 |
This one I'm pleased with
Since playing with the ESP8266 I've been interested in exploring how Serial Flash memory chips could be used for secure data logging on the Micromite as an alternative to SDcards or an external device like an Openlog. These chips are ridiculously cheap so there is no reason not to include one on every PCB. Winbond branded chips work perfectly with the program and are recommended. The example shown above is the one I used to develop the code and will hold 4Mbytes. Compatible chips are available from 512kbyte to 16Mbytes and the code should automatically cater for any of them. Connections to the 8-pin chips are simple: pin 1 (chip select) - connect to any Micromite output capable pin pin 2 (Data out) - connect to the SPI-IN pin on the Micromite pin 3 (write protect) - 10K resistor to 3.3V pin 4 (GND) - connect to ground pin 5 (Data in) - connect to the SPI-OUT pin on the Micromite pin 6 (Clock) - connect to the SPI-CLK pin on the Micromite pin 7 (Hold) - 10K resistor to 3.3V pin 8 (VCC) - connect to 3.3V The chips are a wide version of SOIC and can be easily soldered to a DIP adapter. This could then be socketed on the PCB allowing it to be swapped for reading separately in the same way as an SD card. The intention of the coding was to hide all the workings of the chip from the user and make the user interface as simple as possible so there are just three user routines: formatlog(chip_select_pin_number) This routine completely erases the chip and tests it for correct erasure. In the event that the erase fails the routine will print a message and the program will end. The routine prints the ID of the chip and the size in Mbytes. It also writes to the chip an index of used pages that the logging routine uses to determine where a new record should be appended. writelog(string-to-write, chip_select_pin_number, timestamp) This routine appends the supplied string to the log. If the variable "timestamp" is non-zero it will automatically prepend the date and time comma-separated to the front of the string. If timestamp is 0 or not specified the string will be written as supplied. Logging strings can be constructed in the normal way using STR$ to format numbers and the "+" operator to concatenate multiple sub-strings. readlog(chip_select_pin_number) This routine reads the complete log and prints the contents to the console. To upload to a PC just set Teraterm (or equivalent) to record the terminal session and call readlog. The routine does not change the log data in any way so logging can be restarted after reading without any impact. NB all of the routines are completely self-contained and can be called from the command line with no requirement for anything to have been run or set up previously - there are no global variables or constants - everything required to run is stored on the chip itself. They also obey the requirements for sharing the SPI bus with a TFT screen on the Micromite. This means calls to the formatlog and readlog routines need never be included in the running program as they can just be called from the command line as and when required. It really can't get any easier to include logging in any Micromite program. The test program attached demonstrates just how easy it is to including logging commands in a program. I've tested the code using 1000, 10000, and 100000 log records. It takes 183 seconds to construct and log 10000 records. The C-source is attached, there is nothing complicated and it was all written first in Basic and then just converted to C for improved performance. option explicit option default none dim integer i const chipselectpin=1 'chip select pin ' ' Example program to demonstrate logging to a serial flash chip ' testdata() 'sets up the testdata generator ' formatlog(chipselectpin) 'format the chip ' timer=0 for i=1 to 1000 'log 1000 strings with timestamps writelog(int2Text(i),chipselectpin,1) next i print "1,000 records constructed and logged in ",timer\1000," seconds" pause 3000 ' readlog(chipselectpin) ' read back the log ' end ' '**************************************** ' ' sub formatlog(cspin as integer) 'erase the chip and set up the page index local integer i,bytecount,r(2) local s$ length 10 spi open 20000000,3,8 bytecount = getpagecount(s$,cspin)*256 print "JEDEC ID = ",s$ print "Memory size is ",bytecount\131072," Mbits" if not (erasechip(bytecount,cspin)) then print "erase failed" end endif for i=0 to bytecount\524288 setpagewritten(i,cspin) 'mark the index pages and first real page as used next i spi close print "Format complete" end sub ' sub writelog(s$,cspin as integer,timestamp as integer) 'write a string to the next location on the chip with optional timestamp local integer i,x,buff(63),f,pagecount local d$ length 10 spi open 20000000,3,8 pagecount=getpagecount(d$,cspin) x=getnextfreepage(pagecount,cspin) if x<>pagecount-1 then if x<>pagecount\2048 then x=x-1 'point to the previous page to see if it has space readpage(x,buff(0),cspin) 'read in first page readpage(x+1,buff(32),cspin) 'read in the second page f=getfirstfreebyte(buff()) if timestamp then i=loadstring(date$+","+time$+","+s$,buff(),f) else i= loadstring(s$,buff(),f) endif if i>=256 then 'write the bit on the next page and set the page marker as used writepage(x+1,buff(32),cspin) setpagewritten(x+1,cspin) endif writepage(x,buff(0),cspin) else print "Error: Chip full" endif spi close end sub ' sub readlog(cspin as integer) 'read and print the log local integer x,buff(63),f=0, n,i local s$ spi open 20000000,3,8 x=getpagecount(s$,cspin)\2048 'number of indexpages readpage(x,buff(0),cspin) 'read in first two pages readpage(x+1,buff(32),cspin) if buff(0) =-1 then spi close exit sub 'no data endif n=getstring(s$, buff(0),f) print s$ do while n<>&HFF 'repeat until no more data if f>=256 then x=x+1 readpage(x,buff(0),cspin) 'read in next two pages readpage(x+1,buff(32),cspin) f=f-256 endif n=getstring(s$, buff(0),f) print s$ loop spi close end sub ' function getnextfreepage(pagecount as integer, cspin as integer) as integer 'gets the page number of the next completely unused page local integer i,j,r(31),p=0,found=0,indexsize=pagecount\2048 for i=0 to indexsize-1 'potentially look at all pages readpage(i,r(),cspin) for j=0 to 31 if r(j) then p=j 'there must be at least one spare page if not zero found=1 exit for endif next j if found then exit for next i for j=0 to 63 if (r(p)>>j) and 1 then exit for next j getnextfreepage=i*2048+p*64+j end function ' sub setpagewritten(pageno as integer, cspin as integer) 'set a page as partially or completely written local integer buff(31),i local integer mappage=pageno\2048 'we have 2048 bits per page local integer wordno=(pageno-mappage*2048)\64 'locate the word in the page local integer bitno= 1<<(pageno mod 64) 'locate the bit in the word in the page readpage(mappage,buff(),cspin) buff(wordno) =buff(wordno) XOR bitno writepage(mappage,buff(),cspin) end sub ' CFunction getpagecount 'returns the chip size in 256-byte pages and the chip ID 00000000 27BDFFD0 AFB20020 AFBF002C AFB40028 AFB30024 AFB1001C AFB00018 8CB00000 3C029D00 8C420088 00101880 00621021 8C420000 24030008 10430007 00809021 3C029D00 8C420010 02002021 24050008 0040F809 00003021 3C029D00 8C42001C 02002021 0040F809 24050005 2403009F 3C02BF80 AC435820 3C03BF80 8C625810 30420080 1040FFFD 3C02BF80 8C435820 3C03BF80 AC405820 8C625810 30420080 1040FFFD 3C02BF80 8C535820 3C03BF80 AC405820 8C625810 30420080 1040FFFD 3C02BF80 8C545820 3C03BF80 AC405820 8C625810 30420080 1040FFFD 3C119D00 8E22001C 3C03BF80 02002021 24050006 8C705820 0040F809 00000000 00133A00 24020010 AFA20010 00F43821 8E220030 00073A00 00F03821 26440001 00E03021 0040F809 00073FC3 24040006 A2440000 8FBF002C 2610FFF8 24020001 02021004 00021FC3 8FB40028 8FB30024 8FB20020 8FB1001C 8FB00018 03E00008 27BD0030 End CFunction ' Cfunction erasechip 'size of chip in bytes, chip select pin number 00000000 27BDFFD8 AFB40020 AFBF0024 AFB3001C AFB20018 AFB10014 AFB00010 8CB30000 3C029D00 8C42001C 0080A021 24050005 0040F809 02602021 24030006 3C02BF80 AC435820 3C03BF80 8C625810 30420080 1040FFFD 3C109D00 8E02001C 3C11BF80 02602021 24050006 8E235820 0040F809 00000000 8E02001C 02602021 0040F809 24050005 240200C7 AE225820 3C03BF80 8C625810 30420080 1040FFFD 3C029D00 8C42001C 02602021 24050006 3C03BF80 8C635820 0040F809 3C119D00 3C10BF80 8E220004 0040F809 3404C350 8E22001C 02602021 0040F809 24050005 24020005 AE025820 8E025810 30420080 1040FFFD 00000000 8E025820 AE125820 8E025810 30420080 1040FFFD 00000000 8E125820 8E22001C 02602021 7C129420 0040F809 24050006 32420001 5440FFE6 8E220004 3C029D00 8C42001C 02602021 0040F809 24050005 24030003 3C02BF80 AC435820 3C03BF80 8C625810 30420080 1040FFFD 3C02BF80 8C435820 3C03BF80 AC405820 8C625810 30420080 1040FFFD 3C02BF80 8C435820 3C03BF80 AC405820 8C625810 30420080 1040FFFD 3C02BF80 8C435820 3C03BF80 AC405820 8C625810 30420080 1040FFFD 00000000 8E820004 3C03BF80 8C635820 5C400006 3C02BF80 14400014 3C029D00 8E820000 10400010 3C02BF80 AC525820 3C03BF80 8C625810 30420080 1040FFFD 3C029D00 8C42001C 02602021 24050006 3C03BF80 8C635820 0040F809 00000000 10000007 00001021 3C029D00 8C42001C 02602021 0040F809 24050006 24020001 8FBF0024 8FB40020 8FB3001C 8FB20018 8FB10014 8FB00010 03E00008 27BD0028 End Cfunction ' CSub writepage 'page number, 256 byte data buffer, chip select pin number 00000000 27BDFFD8 AFB3001C AFB00010 AFBF0024 AFB40020 AFB20018 AFB10014 8CD30000 3C029D00 8C42001C 8C940000 00A08021 02602021 0040F809 24050005 24030006 3C02BF80 AC435820 0014A200 3C03BF80 8C625810 30420080 1040FFFD 3C119D00 8E22001C 3C12BF80 02602021 24050006 8E435820 0040F809 00000000 8E22001C 02602021 0040F809 24050005 24020002 AE425820 3C03BF80 8C625810 30420080 1040FFFD 3C02BF80 7E843C00 8C435820 3C03BF80 AC445820 8C625810 30420080 1040FFFD 3C02BF80 7E943A00 8C435820 3C03BF80 AC545820 8C625810 30420080 1040FFFD 3C02BF80 8C435820 3C03BF80 AC405820 8C625810 30420080 1040FFFD 3C02BF80 8C425820 00002021 3C03BF80 240500FF 02041021 80420000 AC625820 8C625810 30420080 1040FFFD 00000000 8C625820 10850003 3C029D00 1000FFF5 24840001 8C42001C 02602021 0040F809 24050006 24120100 3C119D00 3C10BF80 8E220004 0040F809 240400FA 8E22001C 02602021 0040F809 24050005 24020005 AE025820 8E025810 30420080 1040FFFD 00000000 8E025820 AE125820 8E025810 30420080 1040FFFD 00000000 8E125820 8E22001C 02602021 0040F809 24050006 32420001 5440FFE7 8E220004 8FBF0024 8FB40020 8FB3001C 8FB20018 8FB10014 8FB00010 03E00008 27BD0028 End CSub ' CSub readpage 'page number, 256 byte data buffer, chip select pin number 00000000 27BDFFD8 AFB2001C AFB10018 AFBF0024 AFB30020 AFB00014 8CD20000 3C029D00 8C42001C 8C930000 00A08821 02402021 0040F809 24050005 24030003 3C02BF80 AC435820 00139A00 3C03BF80 8C625810 30420080 1040FFFD 3C02BF80 7E643C00 8C435820 3C03BF80 AC445820 8C625810 30420080 1040FFFD 3C02BF80 7E733A00 8C435820 3C03BF80 AC535820 8C625810 30420080 1040FFFD 3C02BF80 8C435820 3C03BF80 AC405820 8C625810 30420080 1040FFFD 3C02BF80 8C425820 00002021 3C03BF80 24050100 AC705820 8C625810 30420080 1040FFFD 00000000 8C705820 02241021 24840001 7C108420 1485FFF6 A0500000 3C029D00 8C42001C 02402021 0040F809 24050006 8FBF0024 8FB30020 8FB2001C 8FB10018 8FB00014 03E00008 27BD0028 End CSub ' CFunction getfirstfreebyte 00000000 90830000 240200FF 00003021 10620014 00003821 90830001 1062000A 24020001 24020002 240500FF 24080100 00821821 90630000 54650006 24420001 10000002 00403021 00403021 10000005 00023FC3 5448FFF7 00821821 24060100 00003821 00C01021 03E00008 00E01821 End CFunction ' CFunction loadstring 'copies a string into the write buffer 00000000 8CC30000 90820000 00621021 0043302A 14C0000A 24660001 00A31821 90870000 24C60001 24C5FFFF 0045282A A0670000 24840001 10A0FFF9 24630001 03E00008 00021FC3 End CFunction ' CFunction getstring ' gets a string ferom the read buffer 00000000 8CC20000 00A21821 90690000 01224821 0122182A 5460000C 25220001 24430001 00A21021 90480000 24630001 2467FFFF 0127382A A0880000 24420001 10E0FFF9 24840001 25220001 00021FC3 ACC30004 ACC20000 00A92821 90A20001 03E00008 00001821 End CFunction ' '**************************** ' test data generation ' 'small sub testdata DATA "one", "two", "three", "four", "five", "six", "seven", "eight", "nine", "ten" DATA "eleven", "twelve", "thirteen", "fourteen", "fifteen", "sixteen", "seventeen", "eighteen", "nineteen" 'tens DATA "twenty", "thirty", "forty", "fifty", "sixty", "seventy", "eighty", "ninety" 'big DATA "thousand", "million", "billion" dim small(19) AS STRING length 10, tens(7) AS STRING length 10, big(2) AS STRING length 10 FOR i = 1 TO 19 READ small(i) NEXT FOR i = 0 TO 7 READ tens(i) NEXT FOR i = 0 TO 2 READ big(i) NEXT ' end sub . FUNCTION int2Text(number AS integer) as string local num AS integer, outP AS STRING length 60, unit AS INTEGER local tmpLng1 AS integer IF 0 = number THEN int2Text = "zero" EXIT FUNCTION END IF num = ABS(number) DO tmpLng1 = num MOD 100 SELECT CASE tmpLng1 CASE 1 TO 19 outP = small(tmpLng1) + " " + outP CASE 20 TO 99 SELECT CASE tmpLng1 MOD 10 CASE 0 outP = tens((tmpLng1 \ 10) - 2) + " " + outP CASE ELSE outP = tens((tmpLng1 \ 10) - 2) + "-" + small(tmpLng1 MOD 10) + " " + outP END SELECT END SELECT tmpLng1 = (num MOD 1000) \ 100 IF tmpLng1 THEN outP = small(tmpLng1) + " hundred " + outP END IF num = num \ 1000 IF num < 1 THEN EXIT DO tmpLng1 = num MOD 1000 IF tmpLng1 THEN outP = big(unit) + " " + outP unit = unit + 1 LOOP IF number < 0 THEN outP = "negative " + outP Do WHILE ASC(RIGHT$(outp,1))<=32 outp = LEFT$(outp,len(outp)-1) loop int2Text = outP END FUNCTION '********************************* #define writeenable 0x06
#define pageprogram 0x02 #define readstatus1 0x05 #define readdata 0x03 #define eraseall 0xC7 #define JEDEC 0x9F #define SPIsend(a) {int j;SPIBUF=a; while((SPISTAT & 0x80)==0); j=SPIBUF;} #define SPIread(a) {SPIBUF=a; while((SPISTAT & 0x80)==0); a=SPIBUF;} #define SPISTAT *(volatile unsigned int *)(0xbf805810) #define SPIBUF *(volatile unsigned int *)(0xbf805820) long long pagecount(char id[],long long *cspin){ int pin=*cspin,i,j,k; if(ExtCurrentConfig[pin]!=EXT_DIG_OUT)ExtCfg(pin,EXT_DIG_OUT,0); PinSetBit(pin,LATCLR); SPIsend(JEDEC); SPIread(i); SPIread(j); SPIread(k); PinSetBit(pin,LATSET); IntToStr(&id[1],i*65536+j*256+k,16); id[0]=6; return (1<<(k-8)); } int erasechip(long long *size, long long *cspin){ int p,pin=*cspin; char m; PinSetBit(pin,LATCLR); SPIsend(writeenable); PinSetBit(pin,LATSET); PinSetBit(pin,LATCLR); SPIsend(eraseall); PinSetBit(pin,LATSET); do { uSec(50000); PinSetBit(pin,LATCLR); SPIsend(readstatus1) SPIread(m); PinSetBit(pin,LATSET); } while(m & 1); PinSetBit(pin,LATCLR); SPIsend(readdata); SPIsend(0); SPIsend(0); SPIsend(0); //start address is zero for(p=0;p<*size;p++){ SPIread(m); if(m!=0xFF){ PinSetBit(pin,LATSET); return 0; } } PinSetBit(pin,LATSET); return 1; } void writepage(long long *address, char d[], long long *cspin){ int k=0,add,m,pin=*cspin; add=*address<<8; //convert page number to byte number PinSetBit(pin,LATCLR); SPIsend(writeenable); PinSetBit(pin,LATSET); PinSetBit(pin,LATCLR); SPIsend(pageprogram); SPIsend((add>>16) & 0xFF); SPIsend((add>>8) & 0xFF); SPIsend(add & 0xFF); for(m=0;m<256;m++) { SPIsend(d[k]); k++; } PinSetBit(pin,LATSET); do { uSec(250); PinSetBit(pin,LATCLR); SPIsend(readstatus1) SPIread(m); PinSetBit(pin,LATSET); } while(m & 1); } void readpage(long long *address, char d[], long long *cspin){ int k,add,pin=*cspin; char m; add=*address<<8; //convert page number to byte number PinSetBit(pin,LATCLR); SPIsend(readdata); SPIsend((add>>16) & 0xFF); SPIsend((add>>8) & 0xFF); SPIsend(add & 0xFF); for(k=0;k<256;k++){ SPIread(m); d[k]=m; } PinSetBit(pin,LATSET); } long long getfirstfreebyte(unsigned char d[]){ int j=0; if(d[0]==0xFF) return 0; for(j=1;j<256;j++){ if(d[j]==0xFF) return j; } return 256; } long long loadstring(unsigned char s[],unsigned char b1[],long long *pos){ int k,j=0,len=s[0],start=*pos; for(k=start;k<=start+len;k++){ b1[k]=s[j]; j++; } return start+len; } long long getstring(unsigned char s[],unsigned char b1[],long long *pos){ int k,j=0,start=*pos,len=b1[start]; for(k=start;k<=start+len;k++){ s[j]=b1[k]; j++; } *pos=start+len+1; //start of next string return b1[start+len+1]; } |
||||