Home
JAQForum Ver 20.06
Log In or Join  
Active Topics
Local Time 21:56 28 Apr 2024 Privacy Policy
Jump to

Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me.

Forum Index : Microcontroller and PC projects : uM2(+): data logging - cheap fast & easy

     Page 5 of 6    
Author Message
centrex

Guru

Joined: 13/11/2011
Location: Australia
Posts: 320
Posted: 02:47pm 02 Jun 2016
Copy link to clipboard 
Print this post

Hi Phil
This total thread is for one particular chip as noted by Matherp on page 1.

It works very well and using the commands and looking at how Matherp loaded the 1000 item data base will give you an idea of what to do. I have it running on the 28 pin micromite and while I dont fully understand the demo programs we are getting there.
Cliff
 
Grogster

Admin Group

Joined: 31/12/2012
Location: New Zealand
Posts: 9063
Posted: 03:10pm 02 Jun 2016
Copy link to clipboard 
Print this post

  centrex said  Does this mean we need the original program for data logging and this new piece if we only want to load or modify specific locations.


I asked matherp for the changes to suit my purposes(and complicate matters), which was to be able to simply page read/write at any location in the chip, at any time, which required his guru C-brain to rustle up the page rewrite sub to read and rewrite the block with the page I wanted to change.

This was MY complication of things, so I would expect that the answer depends on what you want to do.

If you want to just do logging as matherp's original concept stood, then I would stick with his original code examples I would think.

If you want to be able to write any page in the memory, then come along at random and rewrite any page at random(including the same page you just wrote), then you will need the rewrite routine.

matherp will probably explain his concept better then I.

EDIT: @ Phil23 - I can put together an example. How many variables are you wanting to save so I know how many to cater for in the example?Edited by Grogster 2016-06-04
Smoke makes things work. When the smoke gets out, it stops!
 
matherp
Guru

Joined: 11/12/2012
Location: United Kingdom
Posts: 8583
Posted: 09:08pm 02 Jun 2016
Copy link to clipboard 
Print this post

  Quote  Does this mean we need the original program for data logging and this new piece if we only want to load or modify specific locations.


Yes, the original program stands alone. The new CFunction was done for Grogster (but can be generally useful) and allows an individual page to be overwritten. It isn't needed to use the basic logging functionality.

The potential use of this function is that you can save/restore/rewrite three types of data items:

individual strings (must be length 255 which is the default if no length specified)
32-element integer arrays : myarray%(31)
64-element floating point arrays : myarray!(63)

So, if you want to use the code to save/restore individual variables (other than strings) these would need to be copied into arrays.

One of the confusing things is that there are three versions of the code

MX170 SPI
MX170 SPI2 (my implementation)
MX470 SPI2

I could integrate these into a single version but it makes all the CFunctions much bigger so is probably counter-productive.
 
centrex

Guru

Joined: 13/11/2011
Location: Australia
Posts: 320
Posted: 10:38pm 02 Jun 2016
Copy link to clipboard 
Print this post

Thanks Peter
All work very well on the 28 pin MX170, I don't think they need to be integrated into one.

Only my opinion.
Cliff
 
centrex

Guru

Joined: 13/11/2011
Location: Australia
Posts: 320
Posted: 03:24pm 17 Jun 2016
Copy link to clipboard 
Print this post

Hi Matherp

Is it possible to have an 'APPEND' function added to the logging functions.

My reason for this if I am logging at long time intervals say 5mins I would lose a lot of data if it failed before I was able to load a page of data.

If the Function could write the date time immediately and then write a block of data wait for the next block of data to write find the next empty location then write until the page is full and the move to the next page.

Perhaps it can already do this but I cant see how.

Cliff
 
Phil23
Guru

Joined: 27/03/2016
Location: Australia
Posts: 1664
Posted: 06:27pm 17 Jun 2016
Copy link to clipboard 
Print this post

  Grogster said  EDIT: @ Phil23 - I can put together an example. How many variables are you wanting to save so I know how many to cater for in the example?


Sorry Grog,

I only just saw the mention with the example offering.

I'd say only a dozen would do as a start.

Just want to record & save daily maximums & minimums for various temp sensors when I get back to that bit of code.

Too much plumbing, wiring & Dupont pin crimping lately, so didn't see updates to this thread.


Thanks

Phil.
 
matherp
Guru

Joined: 11/12/2012
Location: United Kingdom
Posts: 8583
Posted: 09:18pm 17 Jun 2016
Copy link to clipboard 
Print this post

  Quote  Is it possible to have an 'APPEND' function added to the logging functions.


Not sure what you are asking for.

The original logging functions ONLY support append. No data can be lost as nothing is buffered. When you call writelog it appends the new record immediately.
 
centrex

Guru

Joined: 13/11/2011
Location: Australia
Posts: 320
Posted: 10:45pm 17 Jun 2016
Copy link to clipboard 
Print this post

Hi Peter
As I understand the write page it will write up to 256bytes of data. If I write a string that is only 10bytes long and then another string only 10bytes long these will go on separate pages. To me this means we have lost a large area that could have contained data.

If I am logging with a long delay between readings I could loose a lot of data if I have to wait until the string is 256bytes long should something fail during the logging.

Am I making sense?
Cliff
 
matherp
Guru

Joined: 11/12/2012
Location: United Kingdom
Posts: 8583
Posted: 10:57pm 17 Jun 2016
Copy link to clipboard 
Print this post

Cliff forget the stuff I did for Grogster and go back to the original posts on the first page. This code packs the strings so there is no loss of space and does not buffer. It relies on the fact that you can always change a 1 to a zero. So in your example I write the first 10 bytes, but the next 246 on the page are still all 1s. I can therefore write another string any time after the first by reading that page, updating the next sequence of bytes whilst leaving the first 10 alone and then re-writing it. The code also caters for the write spanning blocks and deals with this as well with no wastage of space.

The writelog function has a few comments to explain how it worksEdited by matherp 2016-06-19
 
centrex

Guru

Joined: 13/11/2011
Location: Australia
Posts: 320
Posted: 11:07pm 17 Jun 2016
Copy link to clipboard 
Print this post

Peter
Thanks I will give it a try, I am using the logging functions.


Cliff
 
centrex

Guru

Joined: 13/11/2011
Location: Australia
Posts: 320
Posted: 11:34pm 17 Jun 2016
Copy link to clipboard 
Print this post

Peter
I have loaded 9 readings as well as the time and date one reading every 5 minutes to a total of 50 bytes ,this is what I get when I readlog()




are you saying that all of these are on one or more pages page.Edited by centrex 2016-06-19
Cliff
 
matherp
Guru

Joined: 11/12/2012
Location: United Kingdom
Posts: 8583
Posted: 11:52pm 17 Jun 2016
Copy link to clipboard 
Print this post

  Quote  are you saying that all of these are on one or more pages page.


????

I'm saying each ascii character in a string uses 1 byte + an overhead of 1 byte for each string logged. The logging code takes care of everything else.
 
centrex

Guru

Joined: 13/11/2011
Location: Australia
Posts: 320
Posted: 12:29am 18 Jun 2016
Copy link to clipboard 
Print this post

OK Peter thanks for your patience.
Cliff
 
Grogster

Admin Group

Joined: 31/12/2012
Location: New Zealand
Posts: 9063
Posted: 03:48am 18 Jun 2016
Copy link to clipboard 
Print this post

  matherp said   Cliff forget the stuff I did for Grogster and go back to the original posts on the first page.


Yes, matherp just did that REWRITE routine for me and anyone else who wanted to page-write, and who would take care of making sure the page was 255 bytes before asking for the write. Ignore all my posts, as matherps original code on page one will do EXACTLY what you are wanting, by appending the data for you with no lost space.

...if I may paraphrase matherp for just a moment.......
Smoke makes things work. When the smoke gets out, it stops!
 
Grogster

Admin Group

Joined: 31/12/2012
Location: New Zealand
Posts: 9063
Posted: 04:07am 18 Jun 2016
Copy link to clipboard 
Print this post

  Phil23 said  
  Grogster said  EDIT: @ Phil23 - I can put together an example. How many variables are you wanting to save so I know how many to cater for in the example?


Sorry Grog,

I only just saw the mention with the example offering.

I'd say only a dozen would do as a start.

Just want to record & save daily maximums & minimums for various temp sensors when I get back to that bit of code.

Too much plumbing, wiring & Dupont pin crimping lately, so didn't see updates to this thread.


Thanks

Phil.


Hey Phil.

The first thing you will need to know, is if you need to use multi-byte words to store the variables, and what kind of data it is. If ALL your variables are going to be numerical and less-then or equal to 256, then it is easy, but if you have larger numbers, you will need to split the value up into multiple 8-bit bytes.

As the SPI memory is an 8-bit device, you cannot store a value greater then 256 in any ONE address, and all addresses will only accept integer values.

If you need to store fractions such as 24.73 degrees or something similar, you can store two bytes for each value. The first is the whole-number degrees, and the second byte is the fraction. Two bytes like that would allow you to record up to 256.256, which would be more then you are ever gonna need for temperature sensors.

Out of interest, and before I get too involved in this idea, is there any reason you cannot use the MM's built-in VAR SAVE command? This will do everything for you, including saving of fractions if you want.

I will need to re-read this thread if you had a reason for NOT using VAR SAVE.
Smoke makes things work. When the smoke gets out, it stops!
 
centrex

Guru

Joined: 13/11/2011
Location: Australia
Posts: 320
Posted: 10:53am 18 Jun 2016
Copy link to clipboard 
Print this post

Hi Grogster
Just to put things straight I was never using the functions that Matherp wrote for you it has always been the logging function.
It is when you do a readlog() that puzzles me because it prints out on separate lines (see the screen dump a couple of posts ago) I am expecting one long comma delimited file or at least a file that is 256 bytes long followed by another similar file.


More reading of how the readlog() works required
Cliff
 
Grogster

Admin Group

Joined: 31/12/2012
Location: New Zealand
Posts: 9063
Posted: 05:02pm 18 Jun 2016
Copy link to clipboard 
Print this post

Okey dokey, understood. I just wanted to make sure, as I was pestering matherp for something a little different then what his original code premise was.
Smoke makes things work. When the smoke gets out, it stops!
 
Phil23
Guru

Joined: 27/03/2016
Location: Australia
Posts: 1664
Posted: 07:44pm 18 Jun 2016
Copy link to clipboard 
Print this post

  Grogster said  
The first thing you will need to know, is if you need to use multi-byte words to store the variables, and what kind of data it is.
[/quote]
Typically 2 digit temperatures, like:-

27.25 etc.
Probably doesn't need 2 decimal precision.

Just 27.2 would suffice.

[Quote]Out of interest, and before I get too involved in this idea, is there any reason you cannot use the MM's built-in VAR SAVE command?[/quote]

Did that for a bit, used VAR SAVE, but I wanted to save the latest MAX/MIN each time it changed & didn't want to wear out the flash.

Lot's of Shutdowns & restarts happening while I'm developing & testing hardware, & didn't want to lose values each time the device is powered off or rebooted.

Even using VAR SAVE once every 5 minutes would have been getting close to 20,000 writes in a day.

I wasn't coding to only save on a change. That seemed difficult with the precision of the Floating variable.

Cheers

Phil





Phil
 
centrex

Guru

Joined: 13/11/2011
Location: Australia
Posts: 320
Posted: 09:24pm 18 Jun 2016
Copy link to clipboard 
Print this post

Hi Phil
Give it a couple of weeks for the slow boat from China to arrive and I will have PCB that could do what you want.
28 pin mite,ILI9341 interface, a position for the memory chip that started these pages as well as a number of other features.
I know it has smd but they are of a size that are reasonably easy to solder, most of it is through hole.
Cliff
 
centrex

Guru

Joined: 13/11/2011
Location: Australia
Posts: 320
Posted: 11:28pm 28 Jun 2016
Copy link to clipboard 
Print this post

Anyone looking for a 28 pin pic that will interface with the ILI9341 touch lcd module, has pads for the Windbond memory chip featured at the top of these pages that runs with Matherp's cfunctions for data logging, it will also interface with the sd card on the lcd again running Matherps cfunctions, provision for Pickit3, I2C and TassyJim and friends wrapper (mmflash)to allow programming of other Pic chips.
I have a few of these left over $2 per board plus postage.
PM me.


Edited by centrex 2016-07-02
Cliff
 
     Page 5 of 6    
Print this page
© JAQ Software 2024