[CALUG] Looking for a tool

[Jim Sansing]

What you are describing is the VSAM file system on the IBM
mainframe MVS operating system.  It has a 'count/key/data'
structure that allows you to find specific variable length records
quickly.  I did a little searching and couldn't find any library
that would support that for you on a Linux/UNIX platform.

However, it shouldn't be too hard to implement.  The first field
of each record is the size, the second is the key, and the third is
the value.  To search the file, you either do a list search--you
don't have to read every byte since you know the size--or you
maintain an index in memory (now you're into sorting for which
I recommend a Red Black tree, see:
  http://web.mit.edu/~emin/www/source_code/index.html).

To do deletes, the brute force method would be to define a
'deleted' key and periodically defragment it by overlaying
deleted records.  If you know the max size of your records,
mmap should do this efficiently with 2 fixed size buffers.

If you think this will require more resources than you have,
any tool will be using at least the same without your knowing
about it.  At least the roll-your-own method gives you control
so you can eliminate unnecessary pieces (otherwise known as
cutting corners ;~).

The only other option I know of for Linux/UNIX is Berkeley DB
which uses flat files.  Subversion uses it, so you could check out
their code as an example.

Later . . .   Jim


Jason C. Miller wrote:
>I'm looking for a UNIX tool to perform a specific set of functionality but I
>dont know if there's anything out there that would do the trick.
>
>I need a very small and light-weight tool that will allow me to me to
>"insert" any number of data into a plain file and provide an interface for
>performing that "insert" and also  "deletes" and "selects"  (using database
>terminology).
>
>Confused yet?  :)
>
>I'm working on an application that has a really small footprint and can't
>afford to run another database intance or provide any more table space in
>any of the already-existing ones.  I'm looking for something that will allow
>me to implement a "/tmp"-like storage facility for miscellaneous data, but
>in a single file.
>
>-------------------------------------------------------------------------------
>EXAMPLE:   (using "STORE" as the storage filename)
>  Say I'm running a session-enabled website and need such a tool for the
>task I'm describing above.  Here's the kind of thing it would do...
>
>$> sometool insert STORE "session id" "103847294837"
>$> sometool insert STORE "number_users" "4"
>$> sometool insert STORE "username" "foo"
>$> sometool insert STORE "privs" "read/write"
>$> sometool insert STORE "admin" "no"
>
>$> SESSION=` sometool select STORE "session_id" `
>$> echo ${SESSION}
>103847294837
>$> USER=` sometool select STORE "username" `
>$> echo ${USER}
>foo
>-------------------------------------------------------------------------------
>
>I need something that's just that simple.  Just FYI...the session-enabled
>website was stricly an example.  It is NOT what I'm working on.  It just
>provided with a valid use-case for the tool.  The solution I'm looking for
>can't be OS-dependent (so Linux or Solaris filesystem tricks in leu of a
>tool are out the window).
>
>Any ideas?  :)
>
>                                                                      -jason
>-------------- next part --------------
>An HTML attachment was scrubbed...
>URL: http://calug.com/pipermail/lug/attachments/20061025/5836528f/attachment.html 
>_______________________________________________
>Columbia, Maryland Linux User's Group (CALUG) mailing list
>CALUG Website: http://www.calug.com
>Email postings to: lug at calug.com
>Change your list subscription options: http://calug.com/mailman/listinfo/lug
>
>