ReadMeImHelpful

The sdaq suite (standard data aq) was designed to be cross platform.
All the main code is written in Perl, GTK is used for the GUI.

It's a lot easier to get going in Linux, even if it's hosted in Virtual Box,
than in windows, as most of the dependencies are either already there in Linux
or getting the missing ones in is easier there.  I am trying to put
most of them in this .zip, but due to size limits in various places, some
will only be on a CD you can get from me with the hardware.
This should also be fairly easy in Macs, since they are also a *nix.
Hopefully a mac user will add to this what's needed there.

You will need Gnuplot, since that's what we use to make the pretty plots
in all opsys.  There are versions precompiled for windows, and for Linux.
I'm using version 4.4, but newer ones (and probably older ones) should also
work - let me know if not.  info@coultersmithing.com is a good way to
get in contact with me.

Perl comes bundled with a bunch of extensions, called modules.  Most of
what this code uses is already there in most Perl distributions.  Any Linux
will already have Perl and most of the required modules.  Windows can
run Perl, but the main distribution that's free is ActiveState, which
has a rather strange way of adding new modules, and which does not have
all of them necessarily.  If you go this route, be sure to install their
version of the package manager so you can get the ones you need integrated
in their system correctly.

Installation:

Linux:
If you haven't already, make a /bin directory under your home, like
/home/doug/bin.  Next time you boot, it will be added to your path automatically.
It then becomes a great place to put all your little runnable utilities, as 
you no longer have to specify a path on the command line.
Copy this .zip file to there, then unpack it. This should create a directory
under bin called SdaqSuite.  This won't yet be on your path, though.
To get this added (in case you're a command line type and don't
just put a launcher in the menubar - which you can't do anyway under
the newer linuxes that have idiotically gone to gnome 3), you
will need to add the directory to the path.  In my systems (Ubuntu 10.04)
this is done by editing the normally hidden file .profile.

Near the end of this there are the lines:

# set PATH so it includes user's private bin if it exists
if [ -d "$HOME/bin" ] ; then
    PATH="$HOME/bin:$PATH"
fi

And you want to add these lines after that:

if [ -d "$HOME/bin/SdaqSuite" ] ; then
    PATH="$HOME/bin/SdaqSuite:$PATH"
fi

This will get these executables in your path next boot.  There are other
scripts in other linuxes that can do the same - your system might
use a different way of setting things up.  rc.d is commonly used on
various distros to do this as well.

Once you have all this "in", you have to mark sdaq and plotdat
as executable.  The easy way is to right click them in nautilus and select
properties, and check the "make file executable" box.  


The way *I* do some of this is with a tool I added to Nautilus, but
there is of course a command line way of doing it to via chmod.
I included the nautilus-scripts add-on in this zip.  To get them in
(it's pretty cool), you unzip them into /home/yourusername/.gnome2/nautilus-scripts
and mark all the scripts as executable.  After that, you can right
click anything in nautilus and have these handy tools available.  Stuff like
command prompt here, root prompt here, gedit as root, things like that
are now easy and quick - you can get the benefit of getting deep into the
directory structure with the GUI, then do command-line kinda things once
you find your spot.  Best of both worlds.

You should initially run sdaq or plotdat from a command line.  If you lack
some of the required dependencies, the errors that will show up there
are actually informative and useful.  You can even copy-paste the names
of the stuff you're missing into say, synaptic package manager or Ubuntu
software center and get them installed till the errors go away.
That is in general easier than doing the old 

unzip
make
sudo make install
routine you'd have to do with the modules I've added to this distribution.
(See CPAN - just Google it, for directions on how to do it the hard way)

Why put all this in a subdirectory under /username/bin?  Because sdaq needs
a place to put its log files, and you probably don't want a ton of those
cluttering up your /bin directory, or alternatively, having to search through
a bunch of other junk in /bin to find them later to plot.  Sdaq and plotdat
assume the log files are in the same directory they are in.

******************************************************************************

Windows:
This section needs work.  Windows needs, on top of the things Linux needs,
the GTK runtime for the gui, and Perl itself.  Due to size limits, I
didn't include gnuplot, Perl for windows with this - just get the latest
and greatest off the web.
Here's where active-state Perl can be found:
http://www.activestate.com/activeperl/downloads
Be sure and get their package manager thing - you may have to click
a checkbox during install to get that, and you'll need it to get and
install any Perl modules you'll be needing.


You'll have to do the same to get a precompiled gnuplot from their site.
Some of the rest of what you'll need is included here.

I've not yet been able to test this on a "clean" windows install, as
all of mine already have a bunch of this stuff on them, so for me, it
just works anyway. I look forward to feedback from anyone who
gets this working on XP or Win7 from scratch so I can fill out this 
section a lot better.

Basically, you'll have to be (or become) a bit more of a power user
than most windows people are to get this going at present.  I've not yet
managed to hornswaggle one of my windows developer pals to write a full
windows "just click yes" installer for all this, but a couple of them
HAVE made this work there.  They just didn't write down all the details
for me, and now they're in the same boat - it works, no error messages to
guide them anymore.

In Linux, files are marked executable, and the first line tells Linux
what program to run them with.  Windows doesn't have this feature.
In general, to get Perl code associated with Perl itself, you'll have
to rename the executables and add .pl to their names to get that
to happen automatically.

--------------------------------------------------------------------------

I also include the pic code for the hardware, if you're adventurous enough
to make your own - this entire project is GPL-V2.  This code builds
in the CCS C compiler in windows (or Linux if you can work out the command
line - CCS supports both opsys, but their IDE only runs in windows).
We are using a PIC - 18f4550, and using the CCS USB driver code more
or less unmodified, except to substitute our name/number for the generic
one they provide.  This is how Linux currently finds the hardware - recent
linuxes create a symlink in /dev/serial/by-id that is our serial port, so
all we have to do is open that.  There's a kludge if perl detects windows
(this uses a different perl module for the serial interface) that goes
off our manufacturer ID, which I hope to change to use the product name
instead, so various things we make won't get the system confused if more
than one is present at a time.

If you do any amount of work with pics - the CCS compiler is just about
the only software I've been happy about paying for, FYI - it's good, it's
stable, it makes very efficient pic code, and they're nice guys there.
In fact, if you don't want to buy our hardware, their dev board/kit
for USB is basically the same as what we provide, sans the input conditioning
electronics and a nice box.  You could easily go with that, adding the input
protection and pullups to the counter inputs, and RC lowpass filters
for the a/d inputs.  It won't be quite as nice, but it should work.  You might
have to unwire some of their built in IO (leds) if they share pins we use.

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

How to run sdaq:

From linux, just type it into a terminal.  EG

>sdaq

On gnome2 machines, you can
create a custom launcher for it and run it from an icon on the menu-bar
as well, it's what I do.  If nothing happens, there were errors, so try
from the command line first to see if everything is kosher.

In windows, get a command prompt and type something like perl sdaq.pl after
renaming it (and you'll have to be in the right directory, or add the install
directory to your windows path, or somewhere in the existing windows
executable path list).

Once it's up, you have some options.  Sdaq is designed to just acquire data.
To this end, the plots are pretty simple.  They'll just show you if your
values are making any sense, there isn't any fancy scaling logic other than
things like counts per second into counts per minute, or a/d values into
volts (only inputs from zero to 5v are good - if you see something clipping
at 5v, you need to go get your inputs scaled better outside our hardware, usually
a simple resistive divider will get it done).  ditto counts, if you're seeing
insane numbers, this will let you know about it so you can fix whatever is
making them insane (usually EMI in the system elsewhere).

The input impedance of the counters is a 47k pullup to 5v, with diodes to
keep in the zero to 5v range.  The input impedance of the a/ds is
bascially infinite for DC, and 47k for HF AC due to a low pass filter we
have in there.  There are no extra protection diodes there, but since there
is 47k in series with each a/d channel, it's going to be pretty hard to fry, and
it's that resistor that will go up in smoke if you put in hundreds of volts.

So, choose any scaling/plotting options you see, and whether to create a log
file or not, and hit the go button down in the lower left to start a run.
If you chose to get a log file (it's not real useful not to, except to test
that your setup is ok), it will go in the same directory with the program, and
its name will be the date-time of the start of the run, with a .log extension.
In other words, no need to provide a unique name for this, we'll figure
that out for you and add this useful info right in the name, as well
as in the first line of the file (so you can rename the file and not lose it).

When you are done, you just hit the same button again to stop it.  Any plots
that were generated will stay on screen till you run again, when they will
be replaced with new ones - so you can stop a run, then study the plots.

This was meant to be really simple to use and the main intent is just to
get your real data onto the disk for later plotting and analysis.

=============================================================================
Plotdat is a program to scatter plot data from log files, currently in
four dimensions - X,Y,Z, and color.  There's a chance we can trick gnuplot
into more - say shape or size for a point, but I've not figured out
how yet.  Their documentation only makes sense in hindsight on some things.
To get the round, colored points for example - the set point style command
is useless, you use the set line-style command instead...had to find that
one out by just trying stuff, so bear with me.

The problem is - there is so much you might want to do to map up to two
counters and up to 8 (currently) a/d inputs onto a plot in meaningful units
I simply couldn't come up with a point-click interface that would handle
all that without cubic miles of edit and check boxes...and I don't have
time to do even that.

So, what I did was to allow you to type in a little perl math type code
into the edit boxes that allow mapping from the raw data into things like
kilovolts, milliamps, pressure, what not - it's on you.

Even this can be a major pain, so I provide for presets to hold any nice
configuration you come up with for reuse later.  To save a preset,
set things up as you like, then type in a name in the preset name box, 
and click "save preset".  A hidden file (on linux, on windows it doesn't
respect the leading . and hide the file) will be created that the program
can reload from later.  Once you click on this (or delete preset) the
program reloads this whole file, and likely as not, will load whatever
preset is first in the perl hash that saves these - not the one you just
saved.  Working on that, but the easy thing to do to get it back is just
use the dropdown and select the one you want.

Perl syntax is almost exactly like C's, Java's or...a lotta other languages.
What is special about perl is it has more operators (even exponentiation),
and that it has enforced "hungarian" notation.  Any scalar variable has a $ in
front of it, or it won't parse.  And that's what we are using here.
For each data point in the log file, my code will grab the two counter
and all the a/d channels into scalars for you, and I take care of the looping
over the whole dataset for you, so in each axis box, you only work with
mapping a single data point.  The GUI contains a bit of a cheat sheet for
all this, but here goes anyway:

Variables:
$x is what will go to the plot program as the X axis data.
$y is the same for the Y axis.
$z is for the Z axis.
$c is color axis.

All of the above must have something put into them or gnuplot will barf.

The source variables are $c0, $c1 (the counter inputs) and $ad0 through
$ad7 for the a/d inputs, which are in a/d counts and are 16 bit numbers that
don't quite have 16 bits of "goodness" since we are using a 10 bit converter,
sampling each channel 2048 times in the second and adding the results, then
right shifting the result into 16 bits.  It IS in general, a lot better
than 10  bits, however - noise acts as a dithering signal, and the 10 bit
a/d has more than 10 bit accuracy on its thresholds.  Call it about 12-13
bits of goodness, even without using the calibration function, if your computer
has a reasonably stable 5v supply on its USB jacks.  It gets better if you
use the $calfactor variable, which is derived from a reference voltage we
put on a/d input 7 in our hardware, it can back out supply voltage errors that
affect the a/d full scale range pretty well.

You are not limited to one line of perl per edit box.  Each legal line
of perl must end with a semicolon (just like C), but there doesn't need
to be a newline between lines unless there is a comment.  Simple, don't use
comments except at the very end.  You could type any legal perl in there - 
even cut-paste from a big program in an editor, it will
work.  So for the life of you (or your disk), don't type in something 
like 'rm -rf';, because it will work!  This is another reason to run this
from a command prompt.  Obviously you will make some errors, and that's where
the helpful error messages will go when you do.

I provide some color maps you can choose from.  The math to generate them
is dauntingly hairy and not obvious how it works, so I simply swiped all
the examples out of the gnuplot documentation.  If you know better ones,
send them to me and I'll include them in the next release.

In general, gnuplot auto-scales everything, so your scaling is mainly
to get the axes lables to make sense, or in the case of signals like
the logged one from the PKR-251 pressure sensor - exponentiate them
and put in the magic numbers to get pressures in whatever unit you
want (this math is in the manual for that).  You almost must use
the calfactor variable for things like this, since exponentiation changes
tiny errors into huge ones.  The PKR signal spans 9 *decades* so getting it
just so is paramount to getting anything meaningful from that.

All the settings are saved in the presets if you like, including axis labels,
color scheme, and all that pesky mapping code I make you put in.

Note, plotdat is designed to ignore blank lines and any line that begins with
a # (comment).  This makes it easy to concatenate runs etc in any text editor.
Time within a run always starts at 1 second, still, but even that has shown
itself to be handy for many cases.

Plotdat will in fact eat any file of the format
(HHHH can be any number of hours from a one digit 0 to millions)

HHHH:MM:SS text number text number text number (and so on forever).

Currently it "knows" that the first two text/number pairs are counter 0 and
counter 1, and all those after that are a/d channels.  It assumes ad7
is connected to a reference voltage (if you use calfactor, that's how
it is calculated).


XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

64 bit.

I've only tried this on linux 64 bit.  The presets don't come over from
32 bit machines correctly, and in my case gnuplot for whatever reason, is
very slow on 64 bit - even on a twice as fast machine, it's maybe 10 times
slower.  I'm going to try a recompile from source code there to see
if that helps - being configured correctly for *that machine* can make a ton
of difference, but it was one heck of a surprise to me.  Everything else
ran just fine.
