Home
JAQForum Ver 24.01
Log In or Join  
Active Topics
Local Time 23:25 13 Mar 2026 Privacy Policy
Jump to

Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me.

Forum Index : Microcontroller and PC projects : First stab at an MMBasic database

Author Message
lizby
Guru

Joined: 17/05/2016
Location: United States
Posts: 3680
Posted: 09:34pm 06 Jan 2026
Copy link to clipboard 
Print this post

The new structures open a lot of opportunities. Users have already suggested one--databases.

I had some fabricated US personal data (500 invented names and locations, but located in accordance with actual demographics). I asked Gemini to convert it into a file for MMBasic on the Pico2 using a structure, and then to produce a simple query program for it to ask for single or AND and/or OR queries.

It's fairly fast--half a second to find 69 records out of 500 which have a state of either MD or NJ; quarter of a second to find the single record with a City of Baltimore and a Zip of 21230.

This is the structure:
Type US500
 DelFlag    As STRING LENGTH 1
 First_Name As STRING LENGTH 12
 Last_Name  As STRING LENGTH 15
 Company    As STRING LENGTH 32
 Address    As STRING LENGTH 33
 City       As STRING LENGTH 21
 County     As STRING LENGTH 22
 State      As STRING LENGTH 4
 ZIP        As STRING LENGTH 5
 Phone1     As STRING LENGTH 14
 Phone2     As STRING LENGTH 14
 Email      As STRING LENGTH 35
End Type


Here's the whole query program:
Option EXPLICIT
Option DEFAULT NONE

' Define Structure
Type US500
 DelFlag    As STRING LENGTH 1
 First_Name As STRING LENGTH 12
 Last_Name  As STRING LENGTH 15
 Company    As STRING LENGTH 32
 Address    As STRING LENGTH 33
 City       As STRING LENGTH 21
 County     As STRING LENGTH 22
 State      As STRING LENGTH 4
 ZIP        As STRING LENGTH 5
 Phone1     As STRING LENGTH 14
 Phone2     As STRING LENGTH 14
 Email      As STRING LENGTH 35
End Type

Dim db As US500

' --- Search Storage ---
' We store "OR Groups". Each group contains a list of "AND Conditions".
' Max 5 OR groups, Max 5 ANDs per group to save memory.
Dim OrGroups$(5) LENGTH 100
Dim INTEGER NumOrGroups

' Temporary parsing variables
Dim rawInput$, currentOr$, currentAnd$, token$
Dim qFld$, qVal$, dbVal$
Dim INTEGER p, i, j, k, orPos, andPos, matchOr, matchAnd
Dim INTEGER count = 0
Dim FLOAT startTime, duration
PRESS ANY KEY ...
>
> list
Option EXPLICIT
Option DEFAULT NONE

' Define Structure
Type US500
 DelFlag    As STRING LENGTH 1
 First_Name As STRING LENGTH 12
 Last_Name  As STRING LENGTH 15
 Company    As STRING LENGTH 32
 Address    As STRING LENGTH 33
 City       As STRING LENGTH 21
 County     As STRING LENGTH 22
 State      As STRING LENGTH 4
 ZIP        As STRING LENGTH 5
 Phone1     As STRING LENGTH 14
 Phone2     As STRING LENGTH 14
 Email      As STRING LENGTH 35
End Type

Dim db As US500

' --- Search Storage ---
' We store "OR Groups". Each group contains a list of "AND Conditions".
' Max 5 OR groups, Max 5 ANDs per group to save memory.
Dim OrGroups$(5) LENGTH 100
Dim INTEGER NumOrGroups

' Temporary parsing variables
Dim rawInput$, currentOr$, currentAnd$, token$
Dim qFld$, qVal$, dbVal$
Dim INTEGER p, i, j, k, orPos, andPos, matchOr, matchAnd
Dim INTEGER count = 0
Dim FLOAT startTime, duration

' ---------------------------------------------------------
' 1. Prompt and Parse (The "Compiler")
' ---------------------------------------------------------
Print "Search DB (Format: Field=Value & Field=Value | Field=Value)"
Line Input "Query: ", rawInput$

If rawInput$ = "" Then End

' Normalize input (Uppercase)
rawInput$ = UCase$(rawInput$)
NumOrGroups = 0

' --- Step A: Split by Pipe '|' (OR Logic) ---
Do While rawInput$ <> ""
 orPos = Instr(rawInput$, "|")

 If orPos > 0 Then
   currentOr$ = Trim$(Left$(rawInput$, orPos - 1))
   rawInput$ = Mid$(rawInput$, orPos + 1)
 Else
   currentOr$ = Trim$(rawInput$)
   rawInput$ = ""
 End If

 If currentOr$ <> "" Then
   NumOrGroups = NumOrGroups + 1
   OrGroups$(NumOrGroups) = currentOr$
 End If
Loop

Print "----------------------------------------------------------------"
Print "DEBUG: Parsed logic:"
For i = 1 To NumOrGroups
 Print "  OR Group " + Str$(i) + ": " + OrGroups$(i)
Next i
Print "----------------------------------------------------------------"

' ---------------------------------------------------------
' 2. Search Loop
' ---------------------------------------------------------
Print "Searching..."
startTime = Timer

Open "us-500.dbt" For INPUT As #1

Do While Not Eof(#1)
 Struct LOAD #1, db

 matchOr = 0 ' Assume the record doesn't match any OR group yet

 ' --- Loop through OR Groups ---
 ' If ANY of these groups evaluates to TRUE, we print the record.
 For i = 1 To NumOrGroups
   currentOr$ = OrGroups$(i)
   matchAnd = 1 ' Assume this specific group is TRUE until an AND fails

   ' --- Loop through AND Conditions inside the group ---
   Do While currentOr$ <> ""
     andPos = Instr(currentOr$, "&")

     If andPos > 0 Then
       token$ = Trim$(Left$(currentOr$, andPos - 1))
       currentOr$ = Mid$(currentOr$, andPos + 1)
     Else
       token$ = Trim$(currentOr$)
       currentOr$ = ""
     End If

     ' Parse Field=Value for this single condition
     p = Instr(token$, "=")
     If p > 0 Then
       qFld$ = Trim$(Left$(token$, p - 1))
       qVal$ = Trim$(Mid$(token$, p + 1))

       ' Check Database Value
       dbVal$ = ""
       Select Case qFld$
         Case "CITY"       : dbVal$ = UCase$(Trim$(db.City, " ", "B"))
         Case "STATE"      : dbVal$ = UCase$(Trim$(db.State, " ", "B"))
         Case "ZIP"        : dbVal$ = UCase$(Trim$(db.ZIP, " ", "B"))
         Case "LAST_NAME"  : dbVal$ = UCase$(Trim$(db.Last_Name, " ", "B"))
         Case "FIRST_NAME" : dbVal$ = UCase$(Trim$(db.First_Name, " ", "B"))
         Case "COMPANY"    : dbVal$ = UCase$(Trim$(db.Company, " ", "B"))
         Case Else         : matchAnd = 0 ' Invalid field fails the group
       End Select

       If dbVal$ <> qVal$ Then
         matchAnd = 0
         Exit Do ' Short circuit: This AND group failed, stop checking it
       End If
     End If
   Loop

   ' If matchAnd is still 1, this OR group passed!
   If matchAnd = 1 Then
     matchOr = 1
     Exit For ' Short circuit: We found a valid OR group, no need to check others
   End If

 Next i

 ' If matchOr is 1, print the record
 If matchOr = 1 Then
   Print db.First_Name, db.Last_Name, db.City, db.State, db.ZIP
   count = count + 1
 End If
Loop

Close #1

duration = Timer - startTime

' ---------------------------------------------------------
' 3. Report
' ---------------------------------------------------------
Print "----------------------------------------------------------------"
Print "Found " + Str$(count) + " records."
Print "Runtime: " + Str$(duration) + " ms"


  Quote  > run
Search DB (Format: Field=Value & Field=Value | Field=Value)
Query: City=Baltimore
----------------------------------------------------------------
DEBUG: Parsed logic:
 OR Group 1: CITY=BALTIMORE
----------------------------------------------------------------
Searching...
Kris            Marrier         Baltimore               MD      21224
Laurel          Reitler         Baltimore               MD      21215
Kaitlyn         Ogg             Baltimore               MD      21230
Eden            Jayson          Baltimore               MD      21202
Izetta          Dewar           Baltimore               MD      21217
----------------------------------------------------------------
Found 5 records.
Runtime: 245.695 ms


I've uploaded the single database file as a zip file: us-500.dbt.

Much more could be done--multiple files, indexes if speed is an issue, forms. I would imagine that 4-8 files with up to 1000 records each would be about as much as anyone would be likely to want on a picomite, especially since you can run SQLite on far more powerful and flexible platforms.

Congratulations to Peter on this fresh inspiration.
us-500.zip
PicoMite, Armmite F4, SensorKits, MMBasic Hardware, Games, etc. on FOTS
 
Amnesie
Guru

Joined: 30/06/2020
Location: Germany
Posts: 746
Posted: 10:34pm 06 Jan 2026
Copy link to clipboard 
Print this post

Amazing all those new opportunities to try new stuff! Wow!
 
lizby
Guru

Joined: 17/05/2016
Location: United States
Posts: 3680
Posted: 11:17pm 08 Jan 2026
Copy link to clipboard 
Print this post

Time for an update. I made some progress yesterday with Gemini, but it was often frustrating. It would forget context, create test data (incorrectly) when I already had test data , make the same errors repeatedly, and make new programs instead of building on the one we were working on.

Six months ago I might have persisted, since the progress was still tangible, but this morning I subscribed to Claude Pro. It has mostly gotten things perfect--the only error it has repeated is putting a DIM were it can be looped to multiple times.

I added a second database for each (fictitious) person in the original one with Age, Occupation, and Salary, plus a link to the original database and Last_Name so I could confirm that I was matching correctly. Then I sorted it on Occupation using STRUCT SORT so the rows wouldn't just line up. With Gemini I got "=", ">", "<" operators working, and "&" (and) and "|" (or) conjunctions, and (with Claude) parentheses to get order correct.

With Claude, the main program can now build an index on any field (with, e.g., "index jobsort on Age"), and can load certain special indices (main table to secondary, i.e. us-500 to jobsort). At this point the program is 872 lines long. All MMBasic, of course.

A search for (State=PA|State=NJ)&Age>49 (that takes a join of the 2 tables) took 1.3 seconds to retrieve 20 records (from 500). After loading the index into memory (with INDEX LOAD jobsort_recno), the same query took .3 seconds.

I should note that Claude added about 600 working lines to a 274-line MMBasic program between 12:45 and 6:15 this afternoon. As I recall, my most productive day, about 40 years ago, was 60 line of complex C code, debugged.

Here's what it looked like:
> run

Enter Query (e.g. Age>39) OR Command (Index tablename on fieldname)
Cmd: load index jobsort_recno
Loaded jobsort_recno index (500 records)

Enter Query (e.g. Age>39) OR Command (Index tablename on fieldname)
Cmd: (state=pa|state=nj)&age>49
Using index: us-500_STATE.ndx
Searching...
--------------------------------------------------------------------------------
Ronny Caiafa, 73 Southern Blvd, PA 19103 54 Engineer
Edna Miceli, 555 Main St, PA 16502 59 Teacher
Elly Morocco, 7 W 32nd St, PA 16502 57 Teacher
Jennie Drymon, 63728 Poway Rd #1, PA 18509 65 Teacher
Loren Asar, 6 Ridgewood Center Dr, PA 18518 54 Developer
Freeman Gochal, 383 Gunderman Rd #197, PA 19320 64 Engineer
Tyra Shields, 3 Fort Worth Ave, PA 19106 63 Nurse
Jacqueline Rowling, 1 N San Saba, PA 16501 65 Manager
Carmen Sweigard, 61304 N French Rd, NJ 08873 51 Sales
Becky Mirafuentes, 30553 Washington Rd, NJ 07062 53 Engineer
Ernie Stenseth, 45 E Liberty St, NJ 07660 64 Teacher
Kimbery Madarang, 798 Lund Farm Way, NJ 07866 54 Engineer
Delisa Crupi, 47565 W Grand Ave, NJ 07105 56 Manager
Candida Corbley, 406 Main St, NJ 08876 62 Developer
Vi Rentfro, 7163 W Clark Rd, NJ 07728 59 Engineer
Nelida Sawchuk, 3 State Route 35 S, NJ 07652 59 Teacher
Tasia Andreason, 4 Cowesett Ave, NJ 07032 51 Developer
Lynelle Auber, 32820 Corkwood Rd, NJ 07104 64 Developer
Alpha Palaia, 43496 Commercial Dr #29, NJ 08003 64 Sales
Selma Husser, 9 State Highway 57 #22, NJ 07306 60 Teacher
--------------------------------------------------------------------------------
20 matches found.
Runtime: 350.966 ms

Enter Query (e.g. Age>39) OR Command (Index tablename on fieldname)
Cmd:


I'm looking forward to seeing regex for STRUCT(FIND ...)
Edited 2026-01-09 10:33 by lizby
PicoMite, Armmite F4, SensorKits, MMBasic Hardware, Games, etc. on FOTS
 
matherp
Guru

Joined: 11/12/2012
Location: United Kingdom
Posts: 11057
Posted: 08:43am 09 Jan 2026
Copy link to clipboard 
Print this post

Excellent work
  Quote  I made some progress yesterday with Gemini, but it was often frustrating.

  Quote  but this morning I subscribed to Claude Pro.

I use AI through the Github Copilot Chat plugin to VSCODE. This give me access to a range of models for a single monthly fee. For simple stuff, I use Claude Sonnet or GPT-5.1-Codex-Max. But for the hard stuff Clause Opus 4.5 is miles ahead of the others but consumes budget much faster with a x3 budget ratio.


Edited 2026-01-09 18:43 by matherp
 
lizby
Guru

Joined: 17/05/2016
Location: United States
Posts: 3680
Posted: 02:06pm 09 Jan 2026
Copy link to clipboard 
Print this post

  matherp said  for the hard stuff Claude Opus 4.5 is miles ahead of the others but consumes budget much faster with a x3 budget ratio.


After 6 hours of work and 600 lines of code added yesterday with Opus 4.5, Claude informed me (at about 6:30pm) that I had run through my budget, and the next refill wouldn't occur until 8pm. So I cooked dinner, ate with my wife, and then idled a bit.

I may not have another day quite as intense or productive as yesterday, but if I am sometimes forced to stop and do other things, that's not such a bad outcome.

The structures I'm using:
Type US500
 DelFlag    As STRING LENGTH 1
 First_Name As STRING LENGTH 12
 Last_Name  As STRING LENGTH 15
 Company    As STRING LENGTH 32
 Address    As STRING LENGTH 33
 City       As STRING LENGTH 21
 County     As STRING LENGTH 22
 State      As STRING LENGTH 4
 ZIP        As STRING LENGTH 5
 Phone1     As STRING LENGTH 14
 Phone2     As STRING LENGTH 14
 Email      As STRING LENGTH 35
End Type

And
Type JobRec
 LinkID     As INTEGER
 LastName   As STRING LENGTH 15
 Age        As INTEGER
 Occupation As STRING LENGTH 20
 Salary     As INTEGER
End Type

And
Type IndexRec
 LinkID     As INTEGER           ' Pointer to Record Position
 StrKey     As STRING LENGTH 50  ' Value for String Sorts (Name)
End Type

PicoMite, Armmite F4, SensorKits, MMBasic Hardware, Games, etc. on FOTS
 
matherp
Guru

Joined: 11/12/2012
Location: United Kingdom
Posts: 11057
Posted: 02:11pm 09 Jan 2026
Copy link to clipboard 
Print this post

  Quote   Claude informed me (at about 6:30pm) that I had run through my budget, and the next refill wouldn't occur until 8pm

There is also a monthly budget which once you hit you are stuffed. Use Sonnet where possible.
 
lizby
Guru

Joined: 17/05/2016
Location: United States
Posts: 3680
Posted: 03:56pm 09 Jan 2026
Copy link to clipboard 
Print this post

Thanks. For anything other than coding on this PicoMite2 database project, I'll use Gemini or ChatGPT5 or Perplexity.
PicoMite, Armmite F4, SensorKits, MMBasic Hardware, Games, etc. on FOTS
 
zeitfest
Guru

Joined: 31/07/2019
Location: Australia
Posts: 662
Posted: 10:42pm 09 Jan 2026
Copy link to clipboard 
Print this post

How will numeric NULL be handled ?
 
lizby
Guru

Joined: 17/05/2016
Location: United States
Posts: 3680
Posted: 11:50pm 09 Jan 2026
Copy link to clipboard 
Print this post

  zeitfest said  How will numeric NULL be handled ?


A good question. Not there yet. But I'll ask Claude.
PicoMite, Armmite F4, SensorKits, MMBasic Hardware, Games, etc. on FOTS
 
lizby
Guru

Joined: 17/05/2016
Location: United States
Posts: 3680
Posted: 12:54am 10 Jan 2026
Copy link to clipboard 
Print this post

MMBasic Database Manager - SQL-like Queries on PicoMite

The database management program is now 1419 lines--via Claude, up from 274 yesterday at noon. The system handles two linked tables with 500 records each and supports a surprisingly rich SQL-like query language. Right now, Claude is costing me $.55 a day for about 600 lines of code a day (but I'm probably soon going to have to pay for additional usage).

Platform: PicoMite Pico2 (RP2350) @ 360MHz
Storage: Flash with binary structure files (.dbt) (SD card optional)
Dataset: 500 fictional U.S. persons (110KB) + employment data (32KB)
Memory: ~264KB free RAM, program uses 39KB (12%)

Database Schema
US-500 Table: First_Name, Last_Name, Company, Address, City, County, State, ZIP, Phone1, Phone2, Email
Jobsort Table: LinkID (foreign key), LastName, Age, Occupation, Salary
Tables are linked via index files, enabling cross-table joins.

Supported Query Features

1. Boolean Queries with AND/OR/Parentheses
age>40
state=CA&occupation=Engineer
age>30|salary>100000
(state=CA|state=NY)&age>50
Last_Name~%pa%
Operators: =, >, <, LIKE (or ~) with % wildcards

2. Query Optimizer (3 Strategies)

Index lookup: Direct access via sorted index files (~350ms)
US-500 first: When query starts with us-500 fields (~730ms)
Jobsort first: When query starts with jobsort fields (~400ms)

System automatically selects optimal strategy based on query structure.

3. Aggregation Functions
COUNT occupation        → Distribution with percentages
SUM salary WHERE age>40
AVG salary WHERE state=CA
MIN age WHERE occupation=Engineer
MAX salary WHERE state=NY
All aggregates support optional WHERE clause for filtering.

4. Pattern Matching (LIKE operator)
state~N%           → Starts with N (NJ, NY, NV, etc.)
Last_Name~%pa%     → Contains "pa"
occupation~%ER     → Ends with ER

5. Indexing System
INDEX us-500 ON state
INDEX jobsort ON occupation
LOAD INDEX jobsort_recno   → In-memory index (3.7x speedup)
Indexes are sorted structure arrays saved as binary files.

Performance Benchmarks

| Operation                | Time        | Notes                          |
|--------------------------|-------------|--------------------------------|
| Simple query (indexed)   | 350ms       | Direct index lookup            |
| Complex query            | 730ms       | Sequential scan with join      |
| LIKE pattern match       | 400-2000ms  | Full table scan                |
| COUNT aggregation        | 320ms       | With REDIM PRESERVE & sort     |
| SUM/AVG/MIN/MAX          | 50-60ms     | Single-pass aggregation        |
| Aggregate with WHERE     | 150-1125ms  | Depends on join complexity     |
| Index creation           | ~2000ms     | One-time operation             |


Development is ongoing.

~
Edited 2026-01-10 11:00 by lizby
PicoMite, Armmite F4, SensorKits, MMBasic Hardware, Games, etc. on FOTS
 
lizby
Guru

Joined: 17/05/2016
Location: United States
Posts: 3680
Posted: 02:23pm 10 Jan 2026
Copy link to clipboard 
Print this post

(Above post written 90% by Claude)

I should note that at present, this is data-retrieval only, though I don't think that updates, adds, deletes for records will be too hard.

The big problem is that the code is highly specific to the two data tables I have. I don't yet see a solution to adding an new table and working with that without having the code rewritten for each table. Strings would be a particular problem, with their varying length.

The goal ultimately would be to have an MMBasic library which approximately replicates SQLite 2.8 (circa 2005). But I'm not sure it will be possible. But it's a good exercise.

Meanwhile, working with a Claude-programmer which is maybe 10 times more productive than I was at my peak is, ummm, interesting.
PicoMite, Armmite F4, SensorKits, MMBasic Hardware, Games, etc. on FOTS
 
homa

Guru

Joined: 05/11/2021
Location: Germany
Posts: 548
Posted: 04:08pm 10 Jan 2026
Copy link to clipboard 
Print this post

Did I understand that correctly? Is it written entirely in Basic? Wow, respect!  

But, as has often been pointed out, wouldn't it make more sense to have a direct implementation in the firmware or as a CSUB? SQLite has already been mentioned several times (https://en.wikipedia.org/wiki/SQLite). Personally, I find SQL incredibly interesting for data loggers, etc.

The suggestion of dynamic extensions seems sensible to me, but I must admit that I have no idea how to write a Basic interpreter and then add extensions to it. However, it's becoming increasingly difficult to squeeze all the different interests into one firmware due to memory limitations.

Nevertheless, I think it's amazing what Peter and you guys are doing here. It's great fun working with the Pico.

Matthias
 
lizby
Guru

Joined: 17/05/2016
Location: United States
Posts: 3680
Posted: 04:35pm 10 Jan 2026
Copy link to clipboard 
Print this post

Yes, entirely in Basic, 1465 lines so far--600 per day added by Claude.

Re memory (RP2350)
> memory
Program:
 38K (11%) Program (1465 lines)
286K (89%) Free

Saved Variables:
 16K (100%) Free

RAM:
 55K (14%) 86 Variables
133K (35%) General
183K (51%) Free

Yes, a CSUB would be the next step if LIBRARY doesn't do it. There's still the issue of how to create a new table which an MMBasic program can then manipulate after giving the LIBRARY/CSUB just the TYPE structure (via, say, CREATE TABLE).
PicoMite, Armmite F4, SensorKits, MMBasic Hardware, Games, etc. on FOTS
 
JohnS
Guru

Joined: 18/11/2011
Location: United Kingdom
Posts: 4281
Posted: 04:57pm 10 Jan 2026
Copy link to clipboard 
Print this post

Why a CSUB?

Would you be happy if it needed considerably more memory?

John
 
matherp
Guru

Joined: 11/12/2012
Location: United Kingdom
Posts: 11057
Posted: 05:46pm 10 Jan 2026
Copy link to clipboard 
Print this post

For the avoidance of doubt, neither this code nor sqlite could possibly be implemented as a cfunction
Edited 2026-01-11 04:03 by matherp
 
lizby
Guru

Joined: 17/05/2016
Location: United States
Posts: 3680
Posted: 10:48pm 10 Jan 2026
Copy link to clipboard 
Print this post

  matherp said  For the avoidance of doubt, neither this code nor sqlite could possibly be implemented as a cfunction


It would be helpful to know your reasoning. The code generated so far by Claude is quite powerful, but also very heavily dependent on the actual data tables. I don't see any clear way to fully generalize--but I'm not very far into the exploration.

That means that Claude could take other tables and replicate what this is doing with another similar program, but not use the code thus far generated to do it.
PicoMite, Armmite F4, SensorKits, MMBasic Hardware, Games, etc. on FOTS
 
matherp
Guru

Joined: 11/12/2012
Location: United Kingdom
Posts: 11057
Posted: 10:51pm 10 Jan 2026
Copy link to clipboard 
Print this post

You would need access to dozens of internal firmware functions and structures and you haven't got them available
 
lizby
Guru

Joined: 17/05/2016
Location: United States
Posts: 3680
Posted: 08:07pm 12 Jan 2026
Copy link to clipboard 
Print this post

MMBasic Database Manager - Weekend Development Summary

Significant additional functionality on Saturday and Sunday:

SHOW command: Selective field display with columnar output, headers, and separators
ORDER BY: Sort results by any field (ASC/DESC)
LIMIT: Restrict number of results displayed
Quoted string values: Handle city names with spaces: city="San Francisco"
DELETE/UNDELETE: Mark records for deletion with full query syntax

 city=Abilene & state=TX DELETE FROM us-500
 city=Abilene & state=TX UNDELETE FROM us-500
 Uses DelFlag field, honors deleted records in all queries

File Output: Export query results to files

TXT format: Columnar formatted output
CSV format: Comma-separated values
Syntax: state=CA SHOW fields TXT filename.txt or CSV filename.csv

BATCH processing: Run multiple commands from file

BATCH filename.txt (Great for testing and automated queries)

Performance Notes:

Indexed queries: ~300-600ms
Non-indexed queries: ~400-900ms
Complex OR queries with parentheses: ~7 seconds (jobsort-first strategy)
All operations stay well within available RAM (264KB free)

Current Feature Set (Complete):
Boolean queries (AND/OR/parentheses)
SQL LIKE pattern matching with % wildcards
Aggregate functions (COUNT, SUM, AVG, MIN, MAX) with WHERE
ORDER BY field [ASC|DESC]
LIMIT n
SHOW field1, field2, ... (columnar output)
DELETE/UNDELETE with query conditions
File output (TXT and CSV formats)
BATCH command for running query files
Indexed queries with 3.7x performance boost
DelFlag honored across all query strategies
Quoted values for strings with spaces

The program is now ~2100 lines and fully functional for interactive database management on the PicoMite (of these particular files)

Sample batch file (test.bat) ran 30 tests successfully, including:

Simple and compound queries
Pattern matching
Aggregates with WHERE clauses
DELETE/UNDELETE operations
File exports
Complex queries with ORDER BY, LIMIT, and SHOW

Total batch runtime: ~60 seconds for 30 diverse queries.

The batch file commands:
# test.bat - Comprehensive database test suite
# Lines starting with # are comments And will be skipped

# Test 1: Simple equality query
city=Abilene

# Test 2: Compound query with And
city=Abilene & state=TX

# Test 3: Query with quoted value(spaces)
city="San Francisco"

# Test 4: OR query
city=Abilene | city=Fresno

# Test 5: Complex query with parentheses
(city=Abilene | city=Fresno) & state=CA

# Test 6: LIKE pattern matching
city~%San%

# Test 7: Numeric comparison
age>60

# Test 8: ORDER BY ascending
state=LA order by age

# Test 9: ORDER BY descending
state=LA order by salary desc

# Test 10: LIMIT results
state=LA limit 5

# Test 11: ORDER BY with LIMIT
state=LA order by salary desc limit 3

# Test 12: SHOW selected fields
state=LA show last_name, first_name, city limit 5

# Test 13: SHOW with ORDER BY And LIMIT
state=TX order by age desc show last_name, age, occupation limit 5

# Test 14: Output To TXT file
state=LA show last_name, first_name, city, age txt Louisiana.txt

# Test 15: Output To CSV file
state=LA show last_name, first_name, city, age csv Louisiana.csv

# Test 16: COUNT aggregate
count state

# Test 17: COUNT on another field
count occupation

# Test 18: SUM aggregate
sum salary

# Test 19: AVG aggregate
avg age

# Test 20: MIN aggregate
min salary

# Test 21: MAX aggregate
max age

# Test 22: Aggregate with WHERE
avg salary where state=CA

# Test 23: Another aggregate with WHERE
max age where occupation=Engineer

# Test 24: DELETE test(mark records)
city=Abilene & state=TX delete from us-500

# Test 25: Verify deletion(should show 0 matches)
city=Abilene & state=TX

# Test 26: UNDELETE test(restore records)
city=Abilene & state=TX undelete from us-500

# Test 27: Verify undelete(should show 2 matches)
city=Abilene & state=TX

# Test 28: Multi-condition with jobsort fields
state=CA & age>50 & salary>100000

# Test 29: LIKE with multiple conditions
city~%Los% & occupation=Manager

# Test 30: LIKE with multiple conditions
city~%Los% & occupation=Engineer

# Test 31: Final complex query
(state=CA | state=TX) & age>50 order by salary desc show last_name, first_name, state, age, salary limit 10


The code is 2137 lines. I ran past Claude's 5-hour limits several times, and paid for extra time ($4.90 USD--much more expensive than if you stay within limits). I am now past my 7-day time until Thursday at 10am, so probably won't continue until then (if at all). If you count $.55 per day for the 7 day period, plus my extra charge, that comes to $8.75. I figure at my last employment, 27+ years ago, at my greatest rate of productivity, my employer would have paid me about $8,000+ for this much code.

This was 4 days work adding to the 274 lines that Gemini had produced.

CAVEAT: The code is entirely specific to the data files used, and I don't see any way to generalize it so that one could feed it a TYPE structure and then run the queries. So I am now using ChatGPT-5.2 to look into converting TinyDB from python to MMBasic. I don't expect the same performance, but may get generalization.


It's been fun. At age 79 and 27 years past retirement I never expected to develop a 2,000-line program again.

~
Edited 2026-01-13 06:25 by lizby
PicoMite, Armmite F4, SensorKits, MMBasic Hardware, Games, etc. on FOTS
 
BarryH

Newbie

Joined: 05/01/2025
Location: Australia
Posts: 29
Posted: 01:00am 13 Jan 2026
Copy link to clipboard 
Print this post

@ Lizby
BarryH
 
Print this page


To reply to this topic, you need to log in.

The Back Shed's forum code is written, and hosted, in Australia.
© JAQ Software 2026