LAN of things NAS and more

[Doug Coulter]

Moving right along, I decided I needed a network-attached storage among other things - the wimpy one provided by my wireless access point wasn't cutting it (1 megabit - over the WIRE!), and didn't do enough.
Pi to the rescue...
While I was at it, I added a few extras, since a pi model 2 is pretty much loafing in this job.
The pi's camera, often handy - this one will be mounted under my eave for a perfect one-way glass so I can see the UPS guy coming and so on - probably get some time-lapse and game pix too.
PHPBB - the same software as used here, why not? It has a lot of uses. While a friend wants me to adopt a configuration management system, PHPBB does all that and more - since it's not web-facing, I set it up to allow any number of any kind of uploads, unlike here, and so as I do things I can keep track of them on threads, with the all-important "what was I thinking" as the text of a post that also has the software uploaded to it. And so on - other things like shopping lists, recipes and general notes on how things work around here, stuff most of you wouldn't be interested in, but I still need it to function with minimum effort.
Of course, this requires "little details" like a web server, in this case NGINX, and Mysql, for which I also added phpmyadmin. I have no idea what I'll wind up doing with this "spare box" on the LAN, someday.
I used vncserver so I can run it headless with a GUI - that'll spoil you fast.

So, I set this pi up with the Hackaday hardware hack (remove r4 and r50 and replace r with a 10k to get full 2 amps out of the USB jacks). I needed that as I'm running a pretty big disk drive off a SATA<>USB cable, and for grins, I also have an arduino uno on there (actually, it's Ada's metro, but basically the same thing) - they seem to come in handy. To make sure I had that power reliably, I used a cheap "simple switcher" I got 10 of from China on ebay for $13.80 - including shipping, and a wall wart. I may later add a gel cel for a ups, and switch down from that instead.
Here's the build - it's going into a cabinet in the wall of my living room, so it doesn't have to be purty:

Just bolt 'em to a plate and screw the plate to the back of the cabinet.

To make this extra-cool and super-maintainable and hopefully more reliable, I did a fairly cool hack that uses that 2 terabyte drive mounted over root - in other words, the SD card is used only at boot, for the UEFI partition at this point.
Even with a mere 5400 rpm laptop drive - it boots faster now than it did off an "extreme" SD card.
This requires a couple tricks. Normally /etc/fstab is used to tell the linux what to mount where. And that matters - so I changed it on this pi. But that's not all - root is special and defined also in the cmdline.txt file in the UEFI partition that's going to be mounted over /boot - and they have to match or this trick won't work.
Here they are for my case:

Code: Select all

proc            /proc           proc    defaults          0       0
/dev/mmcblk0p1  /boot           vfat    defaults          0       2
#/dev/mmcblk0p2  /               ext4    defaults,noatime  0       1
/dev/sda1	/		ext4	defaults,noatime	0	1

# a swapfile is not a swap partition, no line here
#   use  dphys-swapfile swap[on|off]  for that
//192.168.1.240/mnt/	/home/pi/amped	cifs	guest,uid=1000	0	0

In the /etc/fstab above, I simply commented out the original mount of /dev/mmcblk0p2 over root, and substituted /dev/sda1 for that. The other stuff is mounting the usual stuff, plus my WAP share over part of my home directory. Nice to have more than one thing to move stuff around.
/boot/cmdline.txt now looks like this:

Code: Select all

dwc_otg.lpm_enable=0 console=ttyAMA0,115200 console=tty1 root=/dev/sda1 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait

The trick to making this work correctly, other than the above, is that you have to copy all of /dev/mmcblk0p2 to /dev/sda1 *before you reboot*.
This isn't so hard if you do it at the right time.

cd /
sudo cp -rp * /dev/sda1

gets it done.
So, you edit those two files, do that copy (might take awhile) and then reboot - shazam, a pi with 2 terabytes of disk! If you want to be sure it worked, you can then surf to /media, where you should see the old /dev/mmcblk0p2 content - that's where linux puts things it found that have no instructions in /etc/fstab or /boot/cmdline.txt where to mount. If it's not under /media - it's not using the big drive, you have to troubleshoot things.

I had no troubles with mysql or phpmyadmin. NGINX has various issues, which are documented elsewhere on the board here. There's an excellent video tutorial that's a little out of date, and other posts on that thread document what you have to do differently to make it really work . NGINX, once working, is almost certainly the best web server one can use on a pi (if not everything). Mine is doing perl cgi's, php, and all through fast-cgi, so it really sings. PHPBB is faster here from a pi than this website is...go figure.

RPi_Cam_Web_Interface...well, this has a few issues. The author assumes you are a complete noob (perhaps not a totally bad assumption) but gives you no way to put this on top of an existing web server - it will install your choice whether it's there are not already, wipe things out, mess up the configuration files (makes a link to your old sites-available/default but erases the actual file!) - he got ahead of himself trying to look slick with shell scripting and sed?
It is, however some useful stuff once you get over it wiping out your NGINX install and tweaks. I suggest you either put this in first, or backup all your stuff first, and don't use /var/www/html for your own stuff, because it's gonna get wiped out - I put the rest of my stuff at /var/www and let the camera have the /html subdir of that for itself.
To get phpmyadmin and phpbb working, one creates a symbolic link to their "home" and places the link in whatever your web root is. You can then put a link to it in your index.html (or whatever you use for your main home page).
I also added some noob-user type stuff to upload and download files from the subdir I provide for that. There are a few different ways to get to it - you can also mount it directly over someplace on your PC (if you're using the superior linux, anyway) and just use it like any other space - but one that you can only access at around 12 megabytes/second - which is fine for what I use it for.

Before I finish, here's a couple of useful links. Nothing beats the video tutorial on NGINX, plus notes, but this is a good start and helps explain - including some of the other stuff:
https://www.digitalocean.com/community/ ... n-debian-7
This one shows phymyadmin and how to make that symbolic link that makes it all work automagically:
https://www.digitalocean.com/community/ ... emp-server
I used the same trick (with different names, obviously) to link to phpbb.

Here's what my web stuff looks like on the pi - and I didn't even have to disturb the cat on my lap to go over there - I got this shot via vncviewer, as the pi is running headless anyway:

Pi screen showing the web stuff.

And here is some of the content. Since this is in use already, I don't include things like the uploads directory (huge), PHPBB stuff (you can download a fresh one, my content is my business) and so on. Just the cool stuff:

webroot.bz2

Index and upload, with favicon for storage

(1.07 KiB) Downloaded 474 times

And the simple cgi's I'm using for test and file uploading.

cgi.bz2

Some cgi files

(1.73 KiB) Downloaded 534 times

If someone asks, I'll get to uploading the NGINX configs. The only trick not already documented here (somewhere?) is the use of the autoindex in a location directive in an NGINX config to allow browsers to get an FTP like listing of the uploads directory without my having to write any code but that.

This is pretty cool...I haven't used the arduino for anything yet, but if I have enough trouble getting the pi to control the pan-tilt stuff for the camera, well, I already have an arduino sketch for that...who knows, maybe I'll add some other data aq to it.

This doesn't look impressive - I don't care. It does a very worthwhile job, no ifs, ands or buts. You don't look at the home page - you go do things with this, right?

[Jerry]

I have a Netgear NAS, darn thing is almost useless it is so slow when you use it to share a NTFS volume to my Macs. Its because of linux on the NAS. I need to find one of the LaCie NAS that ran Win embedded. NTFS is fine if being served from a PC.

[Doug Coulter]

While NTFS is one of the slower filesystems, I'd bet the cause lies elsewhere. My slow WAP/NAS is slow no matter the filesystem, they just didn't allow its tiny cpu much percentage of cycles/bandwidth for that - it's running linux (probably a super old version) as is the pi - and the pi is fast, no matter the filesystem...I am using ext4 on the pi disk, as that's its native thing, but I've used some of the others in the past - usually something else is the issue if there's a speed problem. The issue with my WAP is that they don't let users (eg me) set anything at all up from their settings webpage - you can't tweak the settings for most of it (people would mostly break it - I understand, but...). With one you make yourself - not a problem. That's why I did this work instead of just buying something off the shelf - I wanted more...and was willing to do the work rather than just splash bucks down and hope.

NTFS has had issues with linux before (but error type stuff like write integrity - now fixed IIUC), but in truth, it shouldn't matter in this case. At any rate, you can use any system the opsys on the share supports - by the time it's shared out, it's not a filesystem anymore anyway, just a service that looks like one.
In the case of SAMBA shares (what most all of them use, including mine) - they are exported as a "windows file system" no matter what the underlying stuff on disk is. Directory listings, file bytes etc are all translated in SAMBA to the SMB protocol, no matter what they originally were.

Of course, it also matters what you define as "slow" - I have gb ethernet (this part is wired) here - and the pi limits to around 12 megabytes/second (yes, faster than its spec of ~100 megabit in actual testing), or about 1/3 of the native disk speed for this disk. That works for me...it's WAY faster than the 100kB/sec of my WAP!

[Jerry]

The linux box served the files fine to a windows machine just reading/writing to it from a mac was excruciatingly slow. I mean only a couple megs per second. Pretty horrible.

[Doug Coulter]

Just sayin - you're likely blaming the opsys for a problem that isn't its fault. When the pi - a pretty minimal machine for this use - is delivering network-limited speeds (it's rated at 100 mbit and easily doing 12 mbyte over that) from a laptop drive, running linux - not a specialized or made-for-purpose reduced-fat version - it ain't the opsys - debian - that's the problem.
Almost zero doubt that my (Amped brand) WAP/NAS runs linux too - heck, they all do. And it's even slower than you are saying yours is, no matter the target device, and it's gigabit on the wire side, and an SSD for the storage (added by me). Obviously they cheaped out on something, or misconfigured it in a way they won't let me fix, for a pi to be kicking its butt totally.

Obviously, if I use a "real" machine for this, since I do have most-all gigabit ethernet - I could go a lot faster - again using the opsys of my choice, google's choice, facebook's choice, amazon's choice - when you gotta do it in bulk, it's linux everywhere these days. Even MS now offers linux on azure because it's (cough) more efficient...

Linux historically didn't support NTFS very well, due to legal hassles microsoft imposed, but those problems weren't mainly speed, they had to do with (ouch, nasty) write integrity, as their support was reverse engineered, and MS kept moving the goalposts. Since I don't have a need to support that, I don't know what the current state is, but over the wire...it's all SAMBA/SMB/NETBIOS no matter what the file or opsys..in other words, the wire format is the same regardless unless you do some very special customizing that won't be compatible with the bulk of clients.

So to sum it up - I'm using a chipset and opsys that didn't make the cut for a cellphone, and beating the pants off a commercial NAS with it (I'm not the only guy to hear and say bad things about the cheap consumer-grade NAS boxes out there). It has to be "something else" wrong with the commercial unit, wouldn't you say? I didn't use anything special, just standard free linux packages, overclock the thing or any of that, and the whole thing draws about 8 watts with the big 2tb drive included.