Thursday, December 13, 2012

How do you know you are doing the right thing?

So... how do you know you are doing the "right thing"?  Aside from feeling that it's the "right thing" (some people think of this as your "gut instinct"), there's not much.  In my families case, things start to go horribly wrong in an attempt to prevent follow-through.  It's usually specific to cars.

My lovely lady and I have decided to adopt (just in case you know of anyone with a child [or one on the way] that we might be able to provide a home for).  We're still in the process of registrations (long process, trust me), but we've committed to the process, no matter what.

And then we hit the "what" :

  • My 2000 Honda Civic refuses to start this week.  Turns out I have a blown head gasket, the crankshaft position sensor is, well, cranky, the engine mounts need to be replaced, and the wheel bearings are starting to wear).  All in all, $350 in parts.  I'm not sure I want to pay labor, so I might end up doing the work myself.
  • The Jeep refused to keep running.  Thought I had the pressure regulator issue worked out, didn't actually have it.  Plus, the headlights were rewired with a relay so that I don't overheat the wire and lose them in the middle of the night while driving, plus getting the door lock rods bent properly so that the window and the door handle works properly without breaking something again.
  • I broke a treadmill.  The plastic hood snapped.  It still runs, but the plastic around the belt wears on the belt.
  • Our furnace starts to show signs of going kaput.  Turns out the flue exhaust motor was starting to break down from overheating (even the fiberglass holding it to the chambers was melted).  It seems to be a recurring problem with Lennox furnaces, but at least I know.  The exhaust motor - $250 (plus labor - they won't sell the part to the public, and I don't blame them because of the liabilities associated with burned gas fumes getting into your heating vents rather then vented outside).
This is not all.  But it's enough for me to stop touching things (because they seem to break).  With some blessings from above, The civic has started (needs about $350 worth of parts, possibly more, waiting to get the Jeep registered), and the Jeep is now running (and should pass emissions when I finally get it over there).  We have the furnace guy coming over tonight....

... well, that was the furnace guy.  The furnace is now in good shape, as is the Heep, uh, Jeep.  The treadmill problem has been identified, and will be resolved later (I can get it to operate, but may just be too busy for that... after all, my soul delights in fatness).  That leaves a Jeep registration and then overhaul the Civic.

On the Corvette front, the wiring has been completed (but not checked).  I must finish cleaning the steering column and then reassemble it before I can test it (the ignition switch and all) to make sure connecting the battery doesn't spark and arc.  Then I can adjust the power windows and get the door panels and dash panels installed.  Probably be the first time in 20 to 25 years that car has had a complete interior in it, but hey, it should like it.

However, it's going to have to take a break and wait for the civic to be rebuilt so that I can have multiple cars again in case the Jeep fails (it's a Jeep - I have to have a full, locked tool box in the back because it's not a matter of if, it's a matter of when).  At that point, I can get back and focus on the 'vette.

Thursday, October 25, 2012

Motion Detection via Flood Light

In our project, the Arduino homebrew security system, we've tried a few variations of motion video capture, Linux Motion, and ZoneMinder.  After a few months of trial and error, I've learned a few things :
  1. Neither package can keep up with recording video at a high enough frame rate (ended up being about 3 frames per second, a far cry from what the camera does with it's 1280x1024 at 15 frames per second, or a standard 30 frames per second at a lower resolution).
  2. Both ZoneMinder and Motion worked the motion detection a little bit differently, and neither one would work for what was needed - a breeze and a tree (or tree shadow) would trigger motion, and relentlessly alert me.  Being desensitized to the motion alerts had the wrong effect, causing it to be worthless for what was intended.
With those two issues, I couldn't use either Motion or ZoneMinder.  I needed something better.  Then I remembered the "anti-cat" video :


In the video, he uses a motion sensor that has an LED that activates when motion is detected.  I'd suggest the PIR sensors - you will have fewer false alarms, and most modern ones are at least IR, so most will do.  Just make sure they have the LED that is active.

I opted for a full flood light.  The problem with these are most have daylight sensors now, and you don't want to the use 120 volt relays on that, because then your motion would only be detected at night.  This is why you need one with an LED.  The LED's are usually active even during daylight hours.

The first step is taking apart the sensor itself, and locating where the LED connects to the circuit board.  Once you have this, you might need to see what kind of voltage is running across those two lines when motion is detected just to be safe.  You will connect those two lines to a relay (why you may need to see the kind of voltage), and that is simply adding a "switch".  You can then wire this into the arduino wiring you used for the doors.

This would give you the following events on the arduino (and the alarm server we coded in http://www.silverhawk.net/2012/04/arduino-assembly-and-testing.html) :
  • MOTION DETECTED
    This event should trigger a motion capture from any video sources you want tied to this camera.
  • ALL CLEAR
    This event should stop the motion capture.  Other possibilities are to cause a temporary sleep (e.g. you want to capture an extra 30 seconds of video before you stop the capture).
Since we had coded the alarm server to be able to call an external command, we can easily interface a tool that immediately starts recording video on a simple command.  The stop is easily done, too, by issuing a command that sleeps and then kills the previous command.

However, there is a problem with that.  If you have multiple sensors tied to a single camera, and one clears (but the other continues to sense motion), your recording would have stopped when the first sensor goes to an all-clear.  The trick here is to use a count of sensors using a camera.  For example, when we start recording, we should open up a count file.  If it contains a 0, then we start the recording.  No matter what it contains, we should increment the count and write it back out.  When we stop recording, we should open the file and decrement the count.  If it's 0 when we're done, kill the recording session.

If you want to get really fancy, you can automatically start transcoding the video and uploading it to an offsite location (very handy, in case someone steals your video server from the house).  Still, I needed a simple hack to get motion detection operational as quickly as I could, and the camera I had included motion detection capabilities using the on-camera motion detection.  That turned out to be the easiest method to use, was the quietest (fewer false positives after tweaking), and had the best quality of video automagically uploaded to the video server.

Monday, September 17, 2012

1/3rd of the car - complete

If I were to section the corvette into three pieces, and then work on each individual piece to completion, the back third would be complete.

Tonight, I finished connecting the fuel sending unit to the wiring harness (I couldn't find those connectors so used some weather, heat-shrink-tubing connectors after using an ohm meter to find which end was which), reconnected the anti-theft horn to the new wiring harness, and completed the power antenna installation (wonder what it will be like to have sound).

This past weekend, I completed the rear storage compartment doors (but not the latches), running wires for the power antenna and amplifier, and ran the wiring for the door lock actuators.

Earlier today, I was able to complete the inputs from the security system into the relays, grounded where neccessary, and ran hot lines where neccessary to complete those. I also finished connecting the neutral safety switch wiring and relay up (the TKO II can't take the amperage that the starter solenoid requires, so I had to use a relay), and finished wiring everything on the stereo connector except for the speakers themselves.

To complete the interior, I have the following tasks remaining :
  1. electrical - The only things remaining on the electrical in the cabin are the security system, and then individual lights. Once complete, the system will need to be verified as operational.
  2. adjust the door glass - Once the electrical has been verified, then I can adjust the window glass. I can't install the door panels until the window glass has been adjusted.
  3. steering column - this needs to be refreshed, and the ignition barrel (the thing the key goes into) needs to be replaced.
  4. dash panels assembled - to finish these, I need to pick up a few parts - a light switch bezel, new duct balls, new center console duct grills, and a speedometer shroud.
  5. install the duct work - (after figuring out where things go, this is going to be fairly simple to accomplish - about 10 minutes of work and it would be done) - unfortunately, this has to wait until the electrical has been completed.
  6. install the panels - This will be the most visually satisfying task. Up to this point, most tasks have been small, and have not had any "visual" progress. Things like soldering, cutting, adjusting, a lot of "behind the scenes" stuff. Installing the panels will be the final step in finishing the interior section of the car.
Once the above is complete, then it will be to the front end to finish that off. To finish the front end, I must do the following :
  1. Connect the fuel lines to the carb (need to feed the double barrel beast).
  2. Connect the dual fans to the wiring harness
  3. Construct/Procure a headlight actuator system, and paint the headlight assemblies to match the vehicle. This is the part that's gonna hurt - the paint is a three part paint, and the color coat is $300 a pint.

Wednesday, August 1, 2012

ZoneMinder vs. motion

In the homebrew security project, we're now to trying to tie video cameras into the system with motion detection. After running ZoneMinder for a week, it has only been capturing at 3 fps. That is utterly awful. I needed something better. I decided to try the older Linux package, "motion". Unfortunately, CentOS doesn't include a pre-packaged RPM, so I have to build it. I grabbed the source from http://sourceforge.net/projects/motion/files/, extracted it (tar -xzf motion-3.2.12.tar.gz), and then ran a simple configure (./configure) just to see if it picked up the ffmpeg libraries any easier than ZoneMinder.

It found everything. I thought, "this is going to be a cinche!" - but after make :
ffmpeg.c: In function ‘fmpeg_open’
ffmpeg.c:365: warning: ‘v_new_stream’is deprecated (declared at /usr/include/ffmpeg/libavformat/avformat.h:1730)
ffmpeg.c:380: error: ‘ODEC_TYPE_VIDEO’undeclared (first use in this function)
ffmpeg.c:380: error: (Each undeclared identifier is reported only once
ffmpeg.c:380: error: for each function it appears in.)
ffmpeg.c:420: warning: ‘v_set_parameters’is deprecated (declared at /usr/include/ffmpeg/libavformat/avformat.h:1766)
ffmpeg.c:446: warning: ‘vcodec_open’is deprecated (declared at /usr/include/ffmpeg/libavcodec/avcodec.h:4155)
ffmpeg.c:501: warning: ‘rl_fopen’is deprecated (declared at /usr/include/ffmpeg/libavformat/avio.h:323)
ffmpeg.c:511: warning: ‘rl_fopen’is deprecated (declared at /usr/include/ffmpeg/libavformat/avio.h:323)
ffmpeg.c:532: warning: ‘v_write_header’is deprecated (declared at /usr/include/ffmpeg/libavformat/avformat.h:1802)
ffmpeg.c: In function ‘fmpeg_close’
ffmpeg.c:599: warning: ‘rl_fclose’is deprecated (declared at /usr/include/ffmpeg/libavformat/avio.h:324)
ffmpeg.c: In function ‘fmpeg_put_frame’
ffmpeg.c:649: error: ‘KT_FLAG_KEY’undeclared (first use in this function)
make: *** [ffmpeg.o] Error 1
Bugger. However, from the dung that I dealt with in ZoneMinder, I specifically remembered that things had to be patched. I didn't see one readily available, so I took the 20 minutes necessary to build one. It's available at http://svn.silverhawk.net/files/motion-3.2.12-ffmpeg-0.10.2-3.patch. That means, here's the process for installing motion :

  1. Download the source for motion from http://sourceforge.net/projects/motion/files/.
  2. Download the patch :
    wget http://www.silverhawk.net/files/motion-3.2.12-ffmpeg-0.10.2-3.patch
  3. Extract the source :
    tar -xzf motion-3.2.12.tar.tg
  4. CD into the source :
    cd motion-3.2.12
  5. Patch :
    patch <../motion-3.2.12-ffmpeg-0.10.2-3.patch
  6. Configure the compile :
    ./configure
  7. Compile :
    make
  8. Install :
    make install
  9. Get a basic configuration :
    cp /usr/local/etc/motion-dist.conf /usr/local/etc/motion.conf

Next up, figure out your camera URL's, and configure the app using the new configuration file /usr/local/etc/motion.conf . Note that this is for a SINGLE camera - for multiples, you will need a separate configuration file for each one! In the configuration file, there are a few settings you should be looking at :
  • process_id_file - if using multiple cameras and multiple files, change this to something with the camera name in it. Otherwise, ensure that this points to an appropriate location on your filesystem. In my case, it defaulted to /var/run/motion/, I reset it to /var/run/ and motion-cameraname (e.g. "/var/run/motion-northside.pid")
  • width - this should be the width of the camera image. For the TrendNet TV-IP322P, this is 1280 (SXGA setting).
  • height - this should be the height. For the TrendNet TV-IP322P, this is 1024.
  • framerate - this should be the rate from your camera. For the TrendNet TV-IP322P, this was 15 (at the SXGA setting).
  • netcam_url - This is the URL to grab an image from your camera. Again, for the TV-IP322P, I used http://ipaddressofcamera/cgi/jpg/image.cgi - I couldn't use the rtsp:// protocol because the motion package doesn't support H264 or MPEG4 (bummer).
  • netcam_userpass - set as "username:password".
  • pre_capture - This is the number of frames before a capture event happens that you want recorded, I set it to 30, or two seconds at our frame rate.
  • post_capture - This is the number of frames after a capture event that you want recorded. I set it to 30 (or two seconds at our frame rate).
  • max_mpeg_time - This is the number of seconds that a generated video file can have (it will start a new one if needed). For this, I set it to 240, or four minutes.
  • output_normal - This tells motion to write individual jpegs or images out. I set this to "off", since I wanted videos.
  • text_right - Since I want to use multiple cameras, I slightly altered this to include the camera name.
  • target_dir - This is where image stills and resulting videos are going to be stored. I tossed this into an apache-enabled directory so that I could easily grab the files from anywhere.
  • webcam_port - I set this to 0 to disable the webcam serving - I didn't want the webcam because I can grab the videos via a Nagios alert link (no need to browse).
  • control_port - I set this to 0 to disable control - the TrendNet TV-IP322P cameras don't have pan and tilt, so it's pointless to try and control it.
  • on_event_start - This is a command line to run when it starts to capture. I used this to run a command that sent a passive check to Nagios stating motion had been detected.
  • on_event_end - This is yet another command line event, but happens when capturing stops. I used this to send the passive check to Nagios to change the status back to "no motion detected, all is OK".
  • on_movie_end - I use this hook to try and maintain the directory. Basically, once a capture stops, it fires a hook to clean out that target directory of anything more than a week old.
  • on_camera_lost - I use this one to fire off another Nagios passive service check indicating we didn't get an image from the camera. This can be used to notify if we lose the camera.
One thing to be aware of is that motion can even be detected when leaves move, or even shadows on concrete change positions. Because of this, another thing to do is to introduce a "mask" for the capture area. To do this :
  1. Grab a frame from the camera (also called an image still - you can get one with the TrendNet TV-IP322P by using the http://ipaddressofcamera/cgi/jpg/image.cgi and saving that image).
  2. Open the saved image in an image editing program (e.g. Gimp), and use the selection tools to surround all areas you wish to be sensitive to movement.
  3. Fill the entire selection with white.
  4. Invert the selection.
  5. Fill the newly selected area with black.
  6. Save the new black-and-white image in a 'pgm' format (e.g. "camera-north.pgm").
  7. Copy the image to the server running motion.
  8. Edit the configuration file for this camera, and find the mask_file option, uncomment and set this to the full path of the image you just created.

Make sure you set motion to start on boot (in case of a power outage), and also make sure you start it up and get the video files, and congrats, video notifications of motion detection!

Wednesday, July 25, 2012

TrendNet TV-IP322P and ZoneMinder

A continuation of our homebrew security project, we visit enabling the camera in ZoneMinder. After a harrowing experience with compiling and installing Zoneminder (I'd almost go with the Linux "motion" application instead), I have the TrendNet TV-IP322P cameras working in ZoneMinder. Once Zoneminder is compiled and running, the next thing is to get the settings right to allow Zoneminder to watch the camera and record events.

This required the following settings for these cameras :
  • General Tab/Name: the name you wish Zoneminder to have for the camera
  • General Tab/Source Type: Remote
  • General Tab/Function: Modetect - this is to allow it to detect motion
  • Source Tab/Remote Protocol: HTTP
  • Source Tab/Remote Method: Simple
  • Source Tab/Remote Host Name: admin:password@ipaddress (example : "admin:mypass@192.168.1.23"
  • Source Tab/Remote Host Port: 80 (unless you've changed the setting in the camera)
  • Source Tab/Remote Host Path: /cgi/jpg/image.cgi
  • Source Tab/Capture Width: 1280
  • Source Tab/Capture Height: 1024
Verifying brought success :
ideventmonitorcausetimedurationframesalarm framestotal scoreavg scoremax score
1NorthWest-1NorthWest Motion07/25 12:46:4213.24 5030206611
2NorthWest-2NorthWest Motion07/25 12:47:046.76244923
3NorthWest-3NorthWest Motion 07/25 12:47:23 11.434424226916
However, after attempting to get ZoneMinder to send immediate notifications to Nagios, and also after finding I was getting 3fps out of ZoneMinder, I opted to try a different method, Linux motion. That was simpler to set up, but two downsides :
  • ZoneMinder allows multiple "zones" or "sensitive areas" on an image stream, motion only allows one (but it can be more complex than the four sides of ZoneMinders).
  • motion doesn't work with multiple cameras very well - you'll have to set up separate motion instances with separate motion.conf files for each one.
So, next up : Linux motion.

Tuesday, July 24, 2012

ZoneMinder CentOS 6.2 64bit

A continuation of our project, homebrew security project, we need to tie in a security camera. I've started trying to install zoneminder in CentOS for the homesecurity project. So far, I haven't been able to locate an RPM for it, but I should be able to use the source. First, grab the source and prepare to compile it :

  1. wget http://www2.zoneminder.com/downloads/ZoneMinder-1.25.0.tar.gz
  2. tar xzvf ZoneMinder-1.25.0.tar.gz
  3. cd ZoneMinder-1.25.0
I kept having problems compiling, so I'll hold off on the configure line that I was using at the time (until after the errors are described). Those errors were :
zm_mpeg.h:63: error: use of enum ‘PixelFormat’ without previous declaration
zm_mpeg.h:64: error: ISO C++ forbids declaration of ‘AVOutputFormat’ with no type
zm_mpeg.h:64: error: expected ‘;’ before ‘*’ token
zm_mpeg.h:65: error: ISO C++ forbids declaration of ‘AVFormatContext’ with no type
zm_mpeg.h:65: error: expected ‘;’ before ‘*’ token
zm_mpeg.h:66: error: ISO C++ forbids declaration of ‘AVStream’ with no type
zm_mpeg.h:66: error: expected ‘;’ before ‘*’ token
zm_mpeg.h:67: error: ISO C++ forbids declaration of ‘AVFrame’ with no type
zm_mpeg.h:67: error: expected ‘;’ before ‘*’ token
zm_mpeg.h:68: error: ISO C++ forbids declaration of ‘AVFrame’ with no type
zm_mpeg.h:68: error: expected ‘;’ before ‘*’ token
zm_mpeg.h:69: error: ISO C++ forbids declaration of ‘uint8_t’ with no type
zm_mpeg.h:69: error: expected ‘;’ before ‘*’ token
zm_mpeg.h:85: error: ‘uint8_t’ has not been declared
In file included from zm_zone.h:27,
from zm_monitor.h:27,
from zmc.cpp:27:
Digging in, I found that my configure was also giving some errors :
checking libavutil/avutil.h usability... no
checking libavutil/avutil.h presence... no
checking for libavutil/avutil.h... no
checking libavcodec/avcodec.h usability... no
checking libavcodec/avcodec.h presence... no
checking for libavcodec/avcodec.h... no
checking libavformat/avformat.h usability... no
checking libavformat/avformat.h presence... no
checking for libavformat/avformat.h... no
checking libswscale/swscale.h usability... no
checking libswscale/swscale.h presence... no
checking for libswscale/swscale.h... no
Just as an FYI, I was using the ffmpeg-devel package from the rpmfusion.org repositories. For those, the include files are in your /usr/include/ffmpeg directory, not /usr/include. Digging into the configure script, I found a few things I could change to make it compile, so now I will provide the configure line :
CPPFLAGS=”-I/usr/include/ffmpeg -I/usr/include” CFLAGS=”-I/usr/include/ffmpeg -I/usr/include” CXXFLAGS=”-D__STDC_CONSTANT_MACROS -I/usr/include/ffmpeg -I/usr/include” ./configure –with-webdir=/var/www/html/zm –with-cgidir=/var/www/cgi-bin –with-libarch=lib64 –with-ffmpeg=/usr –with-webuser=apache –with-webgroup=apache ZM_DB_HOST=localhost ZM_DB_NAME=zm ZM_DB_USER=zmuser ZM_DB_PASS=zmpass –with-extralibs=”-L/usr/lib64 -L/usr/lib64/mysql” ZM_SSL_LIB=openssl
Then I ran into a segmentation fault. The backtrace looked like :
[root@cottonwoodheights ZoneMinder-1.25.0]# gdb zmc
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-50.el6)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
...
Reading symbols from /usr/local/bin/zmc...(no debugging symbols found)...done.
(gdb) run -m 1
Starting program: /usr/local/bin/zmc -m 1
[Thread debugging using libthread_db enabled]
[New Thread 0x7ffff71f6700 (LWP 21926)]

Program received signal SIGABRT, Aborted.
0x00000039b0832885 in raise () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install SDL-1.2.14-3.el6.x86_64 alsa-lib-1.0.22-3.el6.x86_64 bzip2-libs-1.0.5-7.el6_0.x86_64 dbus-libs-1.2.24-5.el6_1.x86_64 enca-1.13-1.el6.x86_64 expat-2.0.1-11.el6_2.x86_64 ffmpeg-libs-0.10.2-3.el6.x86_64 flac-1.2.1-6.1.el6.x86_64 fontconfig-2.8.0-3.el6.x86_64 freetype-2.3.11-6.el6_2.9.x86_64 fribidi-0.19.2-2.el6.x86_64 glibc-2.12-1.47.el6_2.12.x86_64 gnutls-2.8.5-4.el6_2.2.x86_64 gsm-1.0.13-4.el6.x86_64 keyutils-libs-1.4-3.el6.x86_64 krb5-libs-1.9-22.el6_2.1.x86_64 libICE-1.0.6-1.el6.x86_64 libSM-1.1.0-7.1.el6.x86_64 libX11-1.3-2.el6.x86_64 libXau-1.0.5-1.el6.x86_64 libXext-1.1-3.el6.x86_64 libXfixes-4.0.4-1.el6.x86_64 libXi-1.3-3.el6.x86_64 libXtst-1.0.99.2-3.el6.x86_64 libass-0.10.0-1.el6.x86_64 libasyncns-0.8-1.1.el6.x86_64 libcdio-0.81-3.1.el6.x86_64 libcom_err-1.41.12-11.el6.x86_64 libdc1394-2.1.2-3.4.el6.x86_64 libgcc-4.4.6-4.el6.x86_64 libgcrypt-1.4.5-9.el6_2.2.x86_64 libgpg-error-1.7-4.el6.x86_64 libjpeg-6b-46.el6.x86_64 libogg-1.1.4-2.1.el6.x86_64 libraw1394-2.0.4-1.el6.x86_64 librtmp-2.3-3.el6.x86_64 libselinux-2.0.94-5.2.el6.x86_64 libsndfile-1.0.20-5.el6.x86_64 libstdc++-4.4.6-4.el6.x86_64 libtasn1-2.3-3.el6_2.1.x86_64 libtheora-1.1.0-2.el6.x86_64 libusb1-1.0.3-1.el6.x86_64 libuuid-2.17.2-12.4.el6.x86_64 libv4l-0.6.3-2.el6.x86_64 libvorbis-1.2.3-4.el6_2.1.x86_64 libxcb-1.5-1.el6.x86_64 mysql-libs-5.1.61-4.el6.x86_64 nss-softokn-freebl-3.12.9-11.el6.x86_64 openal-soft-1.12.854-1.el6.x86_64 openjpeg-libs-1.3-7.el6.x86_64 openssl-1.0.0-20.el6_2.5.x86_64 pcre-7.8-4.el6.x86_64 pulseaudio-libs-0.9.21-14.el6_3.x86_64 speex-1.2-0.12.rc1.1.el6.x86_64 tcp_wrappers-libs-7.6-57.el6.x86_64 x264-libs-0.120-5.20120303.el6.x86_64 xvidcore-1.3.2-3.el6.x86_64 zlib-1.2.3-27.el6.x86_64
(gdb) bt
#0 0x00000039b0832885 in raise () from /lib64/libc.so.6
#1 0x00000039b0834065 in abort () from /lib64/libc.so.6
#2 0x00000000004142be in Logger::logPrint(bool, char const*, int, int, char const*, ...) ()
#3 0x000000000046d675 in Zone::Load(Monitor*, Zone**&) ()
#4 0x000000000043c78c in Monitor::Load(int, bool, Monitor::Purpose) ()
#5 0x00000000004068a8 in main ()
(gdb)
Splashing a few fprintf(stderr,...) messages throughout the code, I got the zmc command down to the following error:
Zone 1/All for monitor camera-name extends outside of image dimensions, 0, 0, 1279, 1039
Aborted (core dumped)
I added another fprintf(stderr...) line right above the panic line with that error, and pow, I knew what the problem was: the monitor was set up at 1280x1024, and the "Zone" (the motion rectangle) is for 1280x1040. Looking in the DB, yes indeed, the Zone was set badly. I corrected the DB entry, restarted zoneminder, and the camera was STILL showing up in red, meaning it wasn't able to view the camera. So, off to /var/log/messages to see if anything is showing up in there :
Jul 23 12:40:39 security setroubleshoot: SELinux is preventing zmdc.pl from write access on the sock_file zmdc.sock. For complete SELinux messages. run sealert -l 5dde3dda-0d7c-47db-8bfd-5ce7e48fb992 Jul 23 12:40:40 security setroubleshoot: SELinux is preventing uptime from read access on the file /var/run/utmp. For complete SELinux messages. run sealert -l 17270870-69b2-4101-9955-53415acf164f
Hmmm. SELinux again bites me. After a few runs of grabbing data from the audit log (again, the process is "append the AVC error into a separate log file, run audit2allow, enable the module, and try again. Repeat if necessary.") That got me suddenly into the mode where the source is in yellow, not red (meaning the camera is "operational"). But the camera image kept coming up blank. Back to /var/log/messages :
Jul 25 09:42:19 security zmc_m1[2544]: ERR [Error while decoding frame 0]
Jul 25 09:42:19 security zmc_m1[2544]: ERR [256: 00 00 01 b6 51 f4 18 08 85 f7 53 f8 bc 64 17 c1 68 13 c7 79 b1 9c 4d 8c 5c 31 e4 bd 78 38 80 35 18 27 87 60 3a da 9a 69 12 4e a6 db 6b 7b fc ea dd 6b 90 98 f0 8f 13 f3 6b 08 70 22 9a 0a d0 f8 61 dd 6b 84 20 b9 45 ba 06 2c 95 8d 82 41 2e 1c 07 61 42 c2 e7 05 3e 0d 8a eb 0b 86 e7 d2 a2 4c 8c fc 46 50 9b ab b2 26 07 0a 22 67 9c 11 ed a3 e8 d0 1d 45 99 44 36 ea 6b 05 a1 f0 0c 0c 82 be 20 03 8d f3 a0 e2 92 50 e8 59 43 86 87 22 f4 b8 0f 40 18 8e 5b e0 68 32 0a 79 08 23 26 d6 86 29 c9 01 35 c0 ba 43 dc 1e 8d 86 ec 1d 19 f8 44 f6 c1 70 d3 62 10 68 c9 38 38 84 28 f0 bd a0 72 41 9b 70 30 75 01 e1 4b b8 c0 38 ac 88 32 07 15 99 0a 7d d9 37 66 ec 4e 0b b0 70 b4 62 1e 14 8c 4a c2 63 a1 5b 07 42 d1 04 c4 3a 14 0d 7e 87 a0 91 07 06 81 c1 f8 c8 28 b4 20 e3 a1 f8 26 b8 32 08]
Next, I simply tried various configurations until I had one that worked. Next - the configuration for the camera.

Tuesday, July 17, 2012

SELinux/Nagios, and you

I tell ya, selinux almost had me convinced to throw in the towel. I'd find another solution, and then another problem. This went on for the weekend. I didn't throw in the towel, though, because I wanted the system locked down more than the "oh, just disable selinux" crowd would have.

selinux kept preventing nagios from starting up. Here's what I had to ultimately do :
  1. temporarily disable selinux : "setenforce 0"
  2. start nagios : "/etc/init.d/nagios start"
  3. let it run and do it's checks for ten minutes
  4. restart nagios : "/etc/init.d/nagios restart"
  5. stop nagios : "/etc/init.d/nagios stop"
  6. run sealert to build a policy : "sealert -a /var/log/audit/audit.log" - this gave a "catch all" way to create a new module for selinux.
  7. followed the instructions provided by sealert in the previous step on creating and enabling the module.
  8. re-enable selinux : "setenforce 1"
However, there was a problem I just couldn't fix. The disk_check plugin would only work on / - not on /home . I figured this was selinux, but I could never see a message in the usual /var/log/audit/audit.log - I couldn't do the normal "throw the audit log file at 'sealert -a' and build a policy, then use that". Thankfully, a nice website came to the rescue (I'll leave out www.google.com's name to protect the guilty). Though the solution was NOT in the immediate results, tweaking the search request finally yielded something that wasn't NRPE related :

http://edvoncken.net/2012/01/workaround-for-nagios-check_disk-failure-in-rhel-centos-6-2/

My hat is off to this guy. It's a one-line command that fixes the problem.

Finally, the How-To for forked-daapd on CentOS 6.2 64 Bit

FYI, I have superseded this post - I got very frustrated with recompiling the whole kit and caboodle every time I did an update just to have this running.  So, I opted for the mt-daapd server, and it was easier to install, less external dependencies, and still did everything I needed it to.  Try this post before continuing on!

Okay, I'm finally getting to the point of writing up how I finally got forked-daapd installed on my CentOS 6.2 64 Bit media server. It was a pain, but it works.

My process was : try to configure, compile, and install. If it failed, deal with the problem and repeat.

Simple process, but there was a LOT that was problematic.

First, the dependencies (you will need the rpmfusion repository configured) :

yum install gcc glibc-devel avahi-devel ffmpeg-devel libplist-devel taglib-devel flac-devel libunistring-devel java libgcrypt-devel mxml-devel libevent-devel alsa-lib-devel gettext-devel libconfuse-devel gperf zlib-devel ant ant-antlr ant-apache-bcel ant-junit ant-trax antlr tcl-devel git

You will note that we need a stringtemplate RPM as well (hard to find, so I slapped a copy at http://svn.silverhawk.net/files/stringtemplate-3.2.1-3.jpp6.noarch.rpm). I needed to search a bit for one, as it's not in the regular distro, but grab that one.

This should get ALMOST everything. Note that sqlite has to be recompiled. In order to satisfy dependencies, I rebuilt that into an RPM to replace what was there (this one has the option required for forked-daapd). A few others were not found, so I slapped them together, building a number of these RPM's. Currently, the source RPM's are available here :

http://svn.silverhawk.net/files/antlr3-3.1.1-10.el6.src.rpm
http://svn.silverhawk.net/files/libantlr3c-3.1.3-0.2.src.rpm
http://svn.silverhawk.net/files/sqlite-3.6.20-2.el6.src.rpm

You will also want to install the -devel packages for each one once you've built good system packages. You can seamlessly replace any sqlite packages. Note that upgrades might not work with the custom sqlite, but it's easy to grab the source RPM for the upgraded sqlite, add the param, and then upgrade it manually, then the system should update again.

Once those dependencies are installed, you must toss in mxml (available from http://svn.silverhawk.net/files/mxml-2.7.tar (after compiling and installing, copy the mxml.pc file into the appropriate location, mine was /usr/share/pkgconfig/), and then libavl (available from http://alioth.debian.org/~jblache/forked-daapd/libavl_0.3.5.tar.gz) :

  1. Edit the make file, and Comment out "CFLAGS ?= -O2 -fomit-frame-pointer -pipe -mcpu=i686 -w"
  2. Add a new CFLAGS Line :
    CFLAGS = -fPIC -O2 -fomit-frame-pointer -pipe -Wall -g
  3. Replace all "$(LIBRARIES)" variables with "$(LIBRARY)" due to a bug.
  4. Save and Close
  5. Build and install :
    gmake -f GNUmakefile ; gmake -f GNUmakefile install
Now, open the forked-daapd source code, but grab a specific version (you might need to install git if you didn't get it installed earlier) specific for the iTunes 10.5.x functionality (otherwise it will time out) :
  1. mkdir tmp
  2. cd tmp/
  3. git clone https://github.com/CBGoodBuddy/forked-daapd.git
  4. cd forked-daapd
  5. git checkout itunes_v10_5
  6. autoreconf -i
  7. ./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var --enable-flac --enable-musepack --enable-itunes
  8. make
  9. make install
To run the app, first modify your configuration (/etc/forked-daapd.conf), and then add a user :
useradd -s /sbin/nologin -M -r -d /var/cache/forked-daapd -c "iTunes service account" daapd

Friday, June 29, 2012

forked-daapd in CentOS 6.2 64 Bit

Hey, folks; I was setting up forked-daapd in CentOS 6.2 (64 Bit), and ran into a few problems. One of the most recent was that the libantlr3c was not being used (headers were found). I found that if I add a --enable-64bit to the configure line (I used an RPM spec file), it will create it for 64 bit, which is really what I needed. I realized this when it kept failing to configure the forked-daapd with an error about libantlr3 and a missing function, and when I looked in the config.log for the error, the linker was ignoring the 32 bit libraries. (If you haven't enabled 64 bit, it is 32 by default, and you would have to use a 32 bit compiler by default if you wanted to). So, there is another quick hack.

Friday, June 8, 2012

Auto-Documentation - Revisited

Auto-Documentation has been around for quite a while. However, most applications don't cover a wide range of file types (e.g. some are more specific to c/c++, etc). I couldn't get some of the "famous" projects (like "Doxygen") to do exactly what I wanted to do with such a wide range of code types in a lot of my personal SVN repos. So, I did what any red-blooded geek would do. I wrote one. Sure, it still needs some work, but the reality is that it will use Doxygen formats for the C/C++ files to generate documentation, perl (scripts and modules), some of the graphics (and POV-Ray stuff) I have in the repo, and feeds the results through the template-toolkit for the result, and it will do surgical updates just on specific files when something is checked into the repo. Sure, some of you will say "github". But, like the arguments between "RedHat" vs. SuSE, or "Vi" vs. "Emacs", it's a matter of what you cut your teeth on, and SVN (and CVS, used for a decade) still meet my needs perfectly.

Friday, May 18, 2012

For the First Time, A Full Set of Seat Belts

Wow! Last night was a huge milestone in my project! I have both sets of seat belts in the project for the first time since the previous owner had the car!

If you want a little background, I found and procured my 1977 fifteen years ago (about a year or two after I joined the VetteNet). I was ecstatic at the time it showed up. The car barely ran (had to let it warm up for 5 minutes before you could shift into drive or it would die). There was no interior in the car (lots of the original , broken panels), and sometimes, it wouldn't even crank when trying to start it. The body had stress fractures, and the previous owner (who had it for two years) had painted primer over the top of the original lacquer paint. He said he got the car out of a field in Ohio. From my experiences with this car, I believe it sat in that field for 10 years (a literal rat nest in the air cleaner, weeds growing IN the frame, and the frame itself collapsing under its own weight from when I tried to jack it up because of corrosion). It had the original block and TH350 (I still have those) , and the infamous "lazy eye" headlights because of a leak in the reservoir.

It has been a long, hard, educational experience. In the spring of 1995, I saw a gorgeous, victory red '74-'77, and knew I wanted one. I was not a car guy - at the time, I knew they had engines, and wheels, and when you pushed the throttle, a car went. Being in college, the only one I could afford (and I could find) was a "project" that someone was getting rid of. I bought it over the Internet, sight unseen (it supposedly "ran"). I received the car in Dec 1997, and started trying to get it road worthy. For two years, it was the only car I had.

I know many don't appreciate the "Haynes" manuals - I found the manual invaluable with my lack of knowledge. I later picked up an AIM (which I still use). I dismantled the engine twice trying to figure out problems, and realized I had to get this car back into a full, beautiful state if I wanted to drive it. That was in '00 - and the start of a complete rebuild (including removing the frame, rebuilding chunks of the frame and shipping it off to a local company who could spray finishes inside the frame, and putting in a stroker with a Tremec TKO II). Basically, after getting the frame back, I spent the time building a brand new car right beside the pile of parts formerly known as the old one. If I couldn't find a part that I wanted, I rebuilt what I had. Sent it off to the painter in '06 and got it back in '09 (expensive learning experience there - we can leave that out of this conversation though) in non-perfect state, and began to finish putting the thing together. Married in '09, my wife has never seen it run. Prior to me having the car, it's never had both seats in it at the same time.

It is definitely not going to be a top-flighter - I've installed rack and pinion, and a series of other aftermarket parts that make this just the way I like it. It's taken me such a long time, but I am getting closer and closer. There is still a lot of work to do (I want to put in electric, pop-up headlights, and some other goodies for the engine [18*, super, etc]), but I also want to get it on the road. It seems that (once a month) I get an hour to turn a wrench on it, and I make just a little bit of progress. Carpets have been installed for the first time I've had the car, and with the seat belts, I can install quarter panels (still figuring out how they fit into the car - I don't know since I didn't remove them).

But, I'll get it figured out. I'm not even sure I have all of the interior panels - but I'll find out. Has it been a good experience? You bet - I wouldn't trade what I learned for anything. The real problem is that my wife now has things in the queue - the original 1973 Dodge Power Wagon her parents bought new (and she drove to high school), and later she wants me to help her build her own Co*ra kit car. The bug even transferred to my father who purchased a 1959 Pontiac Star Chief to rebuild. Now that I am getting close, I can taste victory, and it is tasting sweet!

Thursday, April 26, 2012

Interrupt-Driven Alarm on Hold - Polling Instead

BUG REPORTS :

Bug # 1 - Polling instead of Interruptions

After finding a bug in the interrupt-driven alarm mechanism, that has been put on hold while the project progresses. The bug is this - with the basic circuit design, if ANY input is open, opening another one will NOT trigger the interrupt.  In lieu of the interrupt-based concepts, I implemented a simpler design that people would have used defacto - polling. In this case, the loop() function sleeps for 50ms, then manually calls the interrupt function, and continues this way indefinitely. Power savings were merely 10 milliamp (very paltry considering the overall design), and simpler is always better. As a result, it's now a polling-based alarm mechanism, and functions well. Code is available at http://svn.silverhawk.net/files/polling_alarm.ino.

Bug # 2 - DD-WRT Router, Static Lease, and Arduino Static Config

This one isn't a bug with the Arduino, but a bug with the system.  In configuring the network, I used my current Asus device with DD-WRT installed, and configured a static lease.  On the Arduino side, I set the IP address statically (no DHCP).  The problem came about after realizing the Arduino could not talk to the outside world and running some iptables routing commands on the DD-WRT :

iptables -t mangle -A POSTROUTING -d 10.1.1.20 -j ROUTE --tee --gw 10.1.1.100
iptables -t mangle -A PREROUTING -s 10.1.1.20 -j ROUTE --tee --gw 10.1.1.100
The above two lines copied all network traffic to and from 10.1.1.20 to my workstation, 10.1.1.100, and then I slapped Wireshark on there.  I saw internal communication from the Arduino to the alarm server, but as soon as it tried to talk externally, nothing got out.  In fact, the Arduino threw "Who as 10.1.1.1?" ARP messages out to the gateway in order to communicate with it, but the gateway (the DD-WRT) simply ignored the Arduino.

Hmph!  I adjusted the static lease in the DD-WRT, and the thing was instantly communicating to the outside world.  It was live (open a door and my VPS server knew what happened).  Awesome!  So, let's run it with the PoE instead of USB power.....

Bug # 3 - Insufficient Power From PoE Device

After having successful communication to the outside world with everything in place (and the sensors installed in the doors), I tried running the Arduino over the PoE with long cables - and some sensors stopped working (they just reported back as "open").  When using the USB to power the device, it worked flawlessly.  Back to the PoE, and they stopped working again.

Conclusion : put the Arduino where it can be powered using a transformer, preferrably a 12v power source.

Next, I tried installing Linux motion after ZoneMinder failed to give me decent enough performance on the frame rate,  but that had poor performance, too, so I opted to hack some security flood lights that had LED indicators - might as well get dual purpose devices.  I start picking the theories for this apart in Motion Detection via Flood Light, and interfacing that to trigger camera recording, not just events.

Friday, April 13, 2012

tcpdump - when no other tools are available to filter a packet

I am fairly used to using tcpdump to capture network data.  However, I really gained the ability to review and understand the captures via Ethereal, later renamed to Wireshark (as it was spun off and the original trademark owner wanted to keep the trademark).  As a systems admin, I have to be familiar with reading the capture file from a network, but not always are the tools available.  As an example, I recently had to filter a packet capture for a partner as we did some troubleshooting, because policies require not exposing internal network structures, layouts, or organization.

So, I found myself one day sitting at my Linux box (without Wireshark installed) having the results of four tcpdumps from separate hosts in separate files.  I needed to combine the captures together and then filter out the partner-specific traffic.  Took me a while to remember the filtering options, so I thought I'd post the process here in the form of a journal - I can't even remember how old I am sometimes.

First, you have to combine the packet capture files together.  You cannot just cat the files together as you might have an incomplete packet from when you started.  The later versions of tcpdump actually have a tool to facilitate this :
mergecap -w silverhawk.cap host1.cap host2.cap host3.cap

After merging the captures together, you will want to remove the stuff not related to your current problem (to prevent the prying eyes from seeing inside of your network).  To do this, take the combined packet capture, and write another one, feeding it through a filter.  Your filters can be complex or simple.  I prefer the simple filters :

tcpdump -r silverhawk.cap -w silverhawk-filtered.cap -n "port 443"
The above would grab all of the TCP port 443 traffic, or the SSL-encrypted traffic (for HTTPS connections).  For most concerns, that alone would be enough (decrypting the SSL traffic would require the SSL certificate keys for the traffic streams you need to see).  However, if you have multiple connections to different port 443 traffic and you would prefer the partner doesn't see whom else you are connecting to, you can do it via IP address :

tcpdump -r silverhawk.cap -w silverhawk-filtered.cap -n "host 200.20.200.20"
Those are fairly simple filters, and should definitely assist in restricting who sees what.  I'd strongly suggest looking at those, and then adjusting as necessary (e.g. adding other, fancier filters by merging them together).

Tuesday, April 10, 2012

Apache, mod_perl, and an example output filter

Perhaps I should explain what an output filter really is.  Before apache serves any content to the web browser, it filters it, or allows different modules to alter it.  Here is an example situation :
You have been given a task where you have HTML pages that contain information for both the authenticated user and the user that hasn't authenticated.  You need to prevent the information intended for the user isn't served when a visitor hasn't authenticated.
How do you implement the above?  It's called an output filter.  For the above example, you will have to implement a simple module that checks a response page for a special tag, check for authentication, and if we are authenticated, allow the contents of the tag to be returned to the browser.  If not authenticated, remove the tag.  As an example, I'll use an HTML type tag :
<AuthenticatedOnly>
You are logged in!
</AuthenticatedOnly>
Please note, we aren't actually implementing the above.  It is merely a hypothetical situation.

There are two ways around creating an output filter.  You can create code that has to be compiled (my mod_template.so is a good example - it's written in C, and it wraps any HTML content in a template so that the whole site appears the same), or you can accomplish the same thing using mod_perl.  We'll take the latter one, as we won't need to install the compiler, just mod_perl.

To start out, we need to modify the Apache configuration to add it in.  Once you have it open, there are three things we need to do :

  1. Enable mod_perl, and insert an "internal" configuration to add our path.
  2. Create the filter module.
  3. Enable the configuration directive for the filter
To enable mod_perl :
LoadModule perl_module modules/mod_perl.so
PerlSwitches -Mlib=/opt/hacks/perl
With those two options in the Apache config, we also need to create our filter.  We'll create it in /opt/hacks/perl, obviously, since we told mod_perl to use that in the above example.


package MyOutputFilter;

use strict;
use warnings;

use base qw(Apache2::Filter);
use Apache2::RequestRec;
use APR::Brigade;
use Apache2::Const -compile => qw(OK);

sub handler : FilterRequestHandler {
  my $f = shift;
  my $bucketBrigade = shift;
  my $r = $f->r;
  my $uri = $r->uri();
  my $contentType = $r->headers_out->get("Content-Type");
  if ($contentType eq 'text/xml') {
    my $content = '';
    my $contentLength = $bucketBrigade->flatten($content);

    # modify $content here

    $f->r->headers_out->set('Content-Length', length $content);
    $f->print($content);
  }
  return Apache2::Const::OK;
}
This is a very simple filter - it does absolutely nothing, but should give you an idea of how a filter works. I'll break things down a little bit at a time. The first line is a good key - it's the name we use later when we enable our specific module. After the name, we have a few libraries we use, and then we must declare our handler function, since that is what mod_perl will call. Ours is the line :
sub handler : FilterRequestHandler {
The contents of that function define what we are doing.

Note that I have grabbed a "bucket brigade". A bucket brigade in apache is a pretty novel idea. Remember the stories about people putting out fires by passing buckets of water between each other? This is the same thing, but the "people" are apache modules, and the "buckets" aren't water, but response data.

In the above example, I call "flatten" on the bucket brigade - for another project, I needed to have ALL of the data at the same time, so it needed to be flattened into one "bucket".  If you work with large data, do NOT flatten them - you can cause Apache to use up free memory!  That assigned the contents of the response into the $content variable, which I can then modify.

After I am done with the data, I can pass it on to the next filter by calling $f->print($content);.  Note that I am also re-setting the Content-Length header - if your browser respects that, and you change the data but don't change the header, you might end up with blank pages (Google Chrome is a good example browser that does that).  Congrats!  You now have a simple filter that you can start to enhance to do some pretty amazing things!

Apache - proxiing to simple proxy to allow viewing of authenticated data

In Apache, we can enable the proxy module to allow our web server to send a request to another location, and we serve the result.  To do this quickly, simply enable the proxy modules you need :
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_connect_module modules/mod_proxy_connect.so
LoadModule proxy_ftp_module modules/mod_proxy_ftp.so
LoadModule proxy_http_module modules/mod_proxy_http.so
Once enabled, you can then create the proxy section. Please note that the following configuration is for an example - do NOT use it without hardening who has access to your proxy!  The config :
<Proxy *>
Order deny,allow
Allow from all
</Proxy>
ProxyPass / http://www.google.com/
Again, note that this will create an open proxy, and that is bad if you don't control who can access what, because you will be the one left holding the bag if they request illegal material. You have been warned!

This works great for most things, but let's presume for a minue that you wanted to serve a page from an HTTPS connection, no matter how it's requested from our web server. We need to enable SSLProxyEngine. If we are using self-signed certificates from that location, you might have to also add "SSLProxyCheckPeerCN off", e.g. :
SSLProxyEngine On
SSLProxyCheckPeerCN off
Let's toss in two more criteria. First, let's presume that you want one subsection of your site to be served from the local filesystem.  We can disable the proxy for a location by adding another proxy pass :
ProxyPass /local/ !
Now, for the second part, let's presume we have a chunk of the proxy that requires authentication.  In this case, we don't want to present the real user with the password pop-up, so, how do we do this?  Web browsers utilize a basic authentication using a series of headers, WWW-Authenticate, and Authorization.  The WWW-Authenticate is a response header that triggers the browser to authenticate the request.  The browser pops up the username/password dialog box, and then uses that information to generate an Authorization header.

Since we really don't need to worry about the WWW-Authenticate header, we can simple inject a permanent Authorization header into the system.  It's done with the RequestHeader directive, similar to :
<Location /svn/trunk/>
RequestHeader set Authorization "Basic dXNlcm5hbWU6cGFzc3dvcmQ="
</Location>
Now, here is a problem. How do we know what to put with the Authorization header?  Looking at the value that follows Basic, it looks like a MIME Base64 encoding.  Feeding it through a simple perl command, we can decode it and find out that it is indeed a Base64 encoding in the format of "username:pass".  We can figure out what we need by installing the MIME::Base64 perl module and running :
[username@hostname ~]$ perl -MMIME::Base64 -e "print encode_base64('username:password');"
dXNlcm5hbWU6cGFzc3dvcmQ=
[username@hostname ~]$
This will inject an authorization header into the proxied request.  Hopefully, this little cheat sheet will be of assistance to someone.

Thursday, April 5, 2012

Arduino - Assembly and Testing

Okay, we have the basics down for how the interface should behave.  Before I continue, I've been asked why not just expose the current states in a web server on the device, so I'd like to take a moment to explain that.  First, there are some things that would be perfect to expose via web service.  Those kinds of states would be temperature or other environmental variables.  The problem with exposing the door sensor via web server is that it would then depend on how often you are probing the actual device via the web.

For example, if you check the device once every minute, you will find that you might not even see the doors "open".  (In the case of an intruder, I doubt they will leave the door open to attract attention while they are in the structure.  Perhaps when they leave, but by then, it could be too late to trigger camera captures or notifications - your valuables could be long-gone.)

Arduino Assembly

The Ethernet shield (W5100) couples with the Arduino Mega 2560 just perfectly.  I suggest that you put them together to run through a test run before soldering on the PoE module.  The shield fits on perfectly by just connecting them :



Once those are connected, go ahead and upload your code to the Arduino.  The next phase for testing is to create the diode board.  I grabbed a cheap prototype board from Radioshack, and just slapped things on there quickly (it might explain the "poor design" in the following photo) :


(Obviously, the diodes are on the other side of the board).  Note that I have soldered ethernet lines up as the switches.  To simplify the other ends of the switches, I use the twisted pairs on a switch basis, e.g. the white-green and green are for the front door, etc.  Those are then tied to a computer header connector (they are cheap, and you can get a male-male header pretty cheap, too - and they work great together!), and also via the diodes to an extra line for the interrupt pin.  Once all of that is soldered together, we connect things up :


We tie our little protoboard to the 5v and ground, and then the individual pins that we want to use as inputs and the interrupt pin.  I then crafted a cheap case (and later found out there are better cases for approximately the same costs, cases that were designed for the Arduino), cut some holes, and slapped it together to check the fit.


Once it's been put together, we take the covers back off, start preparing the configuration for the SD card.  If you are using the code from a previous post, you will create an INI file in the root of the SD card called "ALARM.CFG".  This file has three sections :
[global]

[doors]

[garage]

[network]

[notifications]
The "[global]" section might eventually contain more settings (and did at one point before the code was simplified), but as of now, the only "global" setting is that of the interrupt pin.  For example :

[global]
interruptpin=18
The "[doors]" contains a list of the door switches.  The format for this is a priority:pin - you'll understand the priority when we get to the "[notifications]" section, and the pin is the input on the Arduino that it will be tied to.  These are "open" or "closed" types of switches, so if you were adding window sensors, they'd go right here.  Examples :
[doors]
frontdoor=1:38
backdoor=1:36
gate=1:37
The "[garage]" section contains the garage door switches.  It has the format of priority:pin1,pin2 - again, priority will be discussed later on, but as the garage has four states ("open", "closing", "closed", "opening"), we needed to have two pins (the logic was discussed in the Code Design post).  An example of this :
[garage]
maingarage=1:40,41
The "[network]" contains our configuration for the W5100 card - the IP address we want to assign, gateway, mask, and the MAC address.  For example :
[network]
ipaddr=10.1.0.61
gateway=10.1.0.1
dns=10.1.0.1
mask=255.255.0.0
macaddr=41:4C:41:52:4D:01
The "[notifications]" section contains the places we send state changes for the doors - this happens via a simple network connection (I'll slap some code in here for the server-side, using C for you Linux gurus). The format for these entries in the configuration are priority:IP_address:[EncryptionKey:[port]]
[notifications]
internal network=1:10.1.0.2:11223344556677889900AABBCCDDEEFF
offsite network=1:209.59.216.248:FFEEDDCCBBAA00998877665544332211
Now, to talk about the priority. As an event happens, it has an associated priority with it - e.g. "1" in the above examples. As an event happens associated with a priority, only those notification destinations associated with that priority will be called. It makes it easy to break out various switches to different locations (you can, for example, define the same notification location twice by giving it a different name but the same components, and have two priorities going to the same location). If you don't specify a port, the default port is 3612. If you don't specify an encryption key, you will have a non-encrypted communication. To specify a port without encryption, simply place an entry of the format "Priority:IPaddress::Port" (e.g. no encryption key). The encryption is to prevent a snooper from easily grabbing and spoofing requests to the server. It is a simple encryption, because too much work might cause us to miss other events as they happen if the Arduino is working too hard. The process is a shared key on both ends, and an XOR between them.
void enc(char *src,char *key,int msg_len,int key_len) {
  int   current_pos_key,current_pos_msg;
  current_pos_msg = 0;
  while (current_pos_msg < msg_len) {
    current_pos_key = 0;
    while ((current_pos_key < key_len)
          && (current_pos_msg < msg_len)
          && (src[current_pos_msg + current_pos_key] != '\0')) {
      src[current_pos_msg + current_pos_key] ^= key[current_pos_key];
      current_pos_key++;
    }
    current_pos_msg += key_len;
  }
};
The above is the actual encryption function out of the Arduino code, and gives good idea of how to implement the function.  Call the function again with the same key on the data, and you will have the original message.  For the server-side, the encryption function is identical - we just need the same key configured somewhere with the server.

The entire simple server code is available via http://www.silverhawk.net/files/interrupt_alarm-server-side.c. The tests indeed show that good keys have encrypted communication and that things look great! (I actually tied in the notification server to Google Voice using a fork(), but that code is not available as I'm not sure how far I can go - I was getting door open notifications via SMS!)  Whahooo!  Next up - a bug!

Tuesday, April 3, 2012

Arduino - Design, Code, and Assembly

When I started this component, I went through three different designs :

  1. Learning Curve : With the ultimate objective in mind (and no idea of how to use an Arduino at this point), this project became my Arduino learning curve. In this first "learning" state, it used pull-up or pull-down resistors on a separate circuit board, and the device was actively polling the switches for state changes.
  2. Sleep Mode : As I started to wonder about power consumption, I found that the Arduino utilized interrupts, and (as a result), could be put to sleep. That lead to a redesign using both pull up resistors AND diodes to trigger a separate interrupt pin.  This option was later scrapped due to time constraints and the fact that power consumption dropped to 3mA rather than the original 15mA.  That is NOT a lot of power, and the polling was easier.
  3. Simpler Design : As I progressed in completing the sleep mode, I found that the Arduino INCLUDES on-board pull-up resistors - making them pointless in my initial design. A subsequent test with using the on-board pull-up resistors gave me a fully functional design that required half of the parts I thought I'd end up with.

I toyed with the following schematic, only to find it buggy :


The PROBLEM :

The problem with the above design is that, if any door is closed, the interrupt pin will be grounded out, meaning someone could open any single door, leaving another one closed, and the Arduino would never wake up to find out.  Still, it can be done, I just don't have the time to implement it (using some NEG gates and an OR gate across all switches).  Because it can be done, I'll discuss what was intended with the above.

The THEORY :

All switches (I'm using magnetic reed switches) have one end tied to the ground pin on the arduino. As a switch is closed, it pulls the corresponding digital pin that it is tied to DOWN. (It's high because when we start, we pretend it's an OUTPUT pin and set it to HIGH.) The diode is to prevent switch one from also pulling switch 2 low because we are tying the switches together into an extra pin that is specific for the INT (interrupt). That is required, since we start the Arduino up, initialize everything, then put it to sleep.

Here's how it was supposed to work. When a door is in it's closed state, the switch for that corresponding door is closed, causing the digital pin for that to be pulled low. As a door opens, the switch opens, and the pull-up resistor for that pulls that input to HIGH. It also causes the state to change on the INT. That INT signals the controller to launch a subroutine that scans all of the pins it's configured to looking for a state change, and then launches the appropriate subroutine (e.g. the "closed()" or "closing()" functions).  The interrupt-based code is still available at :

http://svn.silverhawk.net/files/interrupt_alarm.pde


The REAL-LIFE answer : back to "polling"

The solution to the INT problem is to switch back to a "polling" mode.  This means that the Arduino doesn't sleep, but the power savings are negligible. It also means that we call the handleInterrupt() function manually.  The time I delay is 50ms - which is nearly instantaneous to the human mind due to reactions, etc.  It should catch pretty much everything, plus, it allows us to throw in a server (to not just sent notices out, but to be passive in the states, too!), and gives us a little more elbow room.

Onward and Upward :

So, with no interrupts enabled, and with the above description of how it was "supposed" to work, we can surmise that everything pretty much stays the same, but without the INT functionality.  Above, we mentioned "closed", "closing", "opened", and "opening".  You might be asking what the difference is on closed/closing and opened/opening functions. That comes into play with the garage door code. Using a single magnet embedded into the garage door, and two "switches" connected to the endpoints of the door, when the door is closed, one switch will be closed and the other open, and when it opens, those reverse. It also gave the ability to see when both are open, and the last state was "closed", we must logically have started opening the door. The same is also true if both switches are open and the last state is "open" - we are now "closing" the door. That means that the configuration MUST be able to know the difference between a normal switch, and the stateful garage door "switches". For a garage door, the code would look like :
int previous_state;
int state1 = digitalRead(pin1);
int state2 = digitalRead(pin2);
if ((state1 == LOW) && (state2 == LOW)) {
  previous_state = -1;
  eventUnknown("Garage Door Sensors both low! Error");
} else if ((previous_state == 0) && (state1 == HIGH) && (state2 == HIGH)) {
  eventDoorOpening();
} else if ((previous_state == 2) && (state1 == HIGH) && (tstate2 == HIGH)) {
  eventDoorClosing();
} else if (state1 == LOW) {
  eventDoorClosed();
} else if (state2 == LOW) {
  eventDoorOpened();
}
To implement it, you'd need to define a structure that could include this, since the INT specifications mean that the contents are volatile.

For a regular door, it simple checks the previous state, and if the state has changed, call the corresponding "open()" or "close()" function. Also, by passing a structure to a configuration for what sensor we are looking at, we can suddenly reuse the function (memory is limited on an Arduino).

I wanted this device to be configurable, so I had to read from the SD card on an Ethernet shield (W5100). This was done in the setup function. Since I needed a configuration that was easily modifiable without re-uploading the code to the device, I re-invented the INI wheel. The setup function includes the code :
char config_code[10];
char currentChar;
char config_section[80];
char config_line[356];
char *config_key;
char *config_val;
void setup() {
  pinMode(SD_OUTPUT_PIN, OUTPUT);
  if (!SD.begin(SD_PIN)) {
    return;
  }
  // open the configuration file. To write, add FILE_WRITE param
  myFile = SD.open(config_file_name);
  // if the file opened okay, read it. We use an INI-type format
  if (myFile) {
    Serial.print("Reading from ");
    Serial.print(config_file_name);
    Serial.print("...");
    config_line[0] = '\0';
    while (myFile.available()) {
      String cl = myFile.read();
      cl.toCharArray(config_code,10);
      currentChar = atoi(config_code);
      config_line[strlen(config_line)+1] = '\0';
      if (currentChar == '\n') {
        handle_config_line(config_line);
        config_line[0] = '\0';
      }
      config_line[strlen(config_line)+1] = '\0';
      if ((currentChar != '\n') && (currentChar != '\r')) {
        config_line[strlen(config_line)] = currentChar;
      }
    }
    // close the file:
    myFile.close();
    Serial.println("done.");
  } else {
    // if the file didn't open, print an error:
    Serial.print("error opening config file ");
    Serial.print(config_file_name);
    Serial.println("...");
    return;
  }
}
Note that this requires some extra functions (otherwise, setup would have become extremely ugly).
void handle_setting(char *section,char *key,char *val) {
  // put the code in there that would handle your config settings from the INI
};
void handle_config_line(char *line) {
  int  tmp_remove_comment;
  config_val = 0;
  while ((line[strlen(line)-1] == '\n') ||
         (line[strlen(line)-1] == '\r') ||
         (line[strlen(line)-1] == ' ') ||
         (line[strlen(line)-1] == '\t')) {
    line[strlen(line)-1] = '\0';
  }
  tmp_remove_comment = 0;
  while (tmp_remove_comment < strlen(line)) {
    if (line[tmp_remove_comment] == ';') line[tmp_remove_comment] = 0;
    tmp_remove_comment++;
  }
  if (strlen(line) > 0) {
    if (line[0] == '[') {
      strcpy(config_section,line + 1);
      while (config_section[strlen(config_section)-1] == ']') {
        config_section[strlen(config_section)-1] = '\0';
      }
    } else {
      config_key = line;
      config_val = line;
      while ((config_val[0] != '\0') && (config_val[0] != '=')) {
        config_val++;
      }
      if (config_val[0] != '\0') {
        config_val[0] = '\0';
        config_val++;
        handle_setting(config_section,config_key,config_val);
      }
    }
  }
}
Polling doesn't require the following chunk of code! If we get a good interrupt design and need to implement, then, while we are in the setup function, we need to set up the interrupt pins. We do this by enabling the pull-up resistor on the interrupt pin, and then attaching the interrupt function (handleEvent) to that pin :
// set pull-up resistor
digitalWrite(interruptPin,HIGH);
// attach the interrupt
switch (interruptPin) {
  case 2 :
    attachInterrupt(0,handleEvent,CHANGE);
    break;
  case 3 :
    attachInterrupt(1,handleEvent,CHANGE);
    break;
  case 18 :
     attachInterrupt(5,handleEvent,CHANGE);
    break;
  case 19 :
    attachInterrupt(4,handleEvent,CHANGE);
    break;
  case 20 :
    attachInterrupt(3,handleEvent,CHANGE);
    break;
  case 21 :
    attachInterrupt(2,handleEvent,CHANGE);
    break;
};
For each switch, we also need to enable the pull-up resistors :
digitalWrite(pin1,HIGH);
pinMode(pin1,INPUT);
If we were doing the interrupt driven, we'd have to put the thing to sleep. This makes the main loop() function VERY small and short. In fact, the loop() function is ONLY powering the device down :
void loop() {
  set_sleep_mode(SLEEP_MODE_PWR_DOWN);
  sleep_enable();
  sleep_mode();
  sleep_disable();
}
Since we are not using interrupts (we're polling, remember?), our loop function will call our "handleEvent()" function manually by doing the following :
void loop() {
  delay(50ms);
  handleEvent();
}
The interrupt function (used in both interrupt and polling code sets) is fairly simple :
void handleEvent() {
};
At that point, it's merely a matter of making the handleEvent function check all of the pins it needs to. To see the full code, check out http://svn.silverhawk.net/files/polling_alarm.ino. Next, we put it all together to test in Arduino - Assembly and Testing.

Introducing Project Homebrew Security System

The Arduino home security system has led into a new project.  A year ago, I had acquired some outdoor cameras from Harbor Freight for relatively cheap costs ($40 for a color camera).  At the time, I had a PVR device (home built DVR system to record on two channels), and enabled the "motion" application on Linux to work with them.  For a simple and cheap "security" system, it sufficed, but it was isolated to a single room, and it wasn't very "secure", and was fairly limited (the requirements list, points one and two).

After purchasing a home, I realized that I needed something a little better.  I then purchased a 4 channel recorder from a maker.  That device filled the need (almost).  The device kept freezing every few months, without so much of a notification (point 3).  It did show when someone broke into the car, but didn't provide enough clarity (point 4 in the requirements).  The device primarily was for a video stream, and the search mechanism was pretty poor when looking for video files (leading to requirement 5).  Though the device handled input for "GPIO's" (general input pins), there were no instructions on how to use them.

I picked up an Arduino (my first Arduino project), and thought about using it as a "sensor" for doors being opened.  I realized that there were some added requirements.  Those requirements culminated into the following list :

The things learned so far (e.g. the requirements of the system) :
  1. An external alert mechanism
  2. Handle more than two video streams
  3. Stability
  4. Clarity on video streams, both night and day
  5. Good interface
  6. Able to handle external notifications (e.g. a door, gate, motion, heat, or seismic sensor)
  7. Logs for events, from cameras to sensors
  8. Able to send notifications to an off-site location
  9. Nothing "wireless" - wireless is easily spoofed and jammed, and is easily eavesdropped on
  10. Expandable (e.g. connecting in to phone systems via Asterisk)
  11. System monitoring
The above list of requirements gave me solid ground to start working from.  First, since I wanted a system that was stable, I needed a piece of hardware that would not have to worry about doing the device capturing - so I knew I needed IP cameras (or "network cameras").  Because I didn't want to run power lines for each camera, that meant these IP cameras needed to have PoE (power-over-ethernet), and I wanted an effective 40 feet range at night (to cover automobiles, fences, etc).  I finally did find the cameras I wanted : TRENDnet TV-IP322P cameras.  I already have lines run for the smaller cameras, so I will use those to "fish" the CAT cables through the walls.

I was able to design a cheap piece of hardware to cover point 6 in the above.  That was the Interrupt-Driven Arduino Alarm.  I've since expanded that project into the above requirements list, and will document the design and implementation of the system through a few different blog posts.  Additionally, due to time constraints and a bug found in that interrupt-driven code for the Arduino, it is no longer interrupt-driven, but a polling mechanism.  Reasons will be identified in the design post.  However, the parts list so far :
  1. TRENDnet TV-IP322P cameras, IP-based, PoE, night vision to 20m, good resolution (1280x1024), cost : $420 each
  2. TRENDnet TPE-S44 8-port, 100Mbps, PoE switch [enough ports for all cameras, arduino, LAN link, and local device to handle video capture] ($50.00)
  3. Arduino Mega 2560 ($48.00)
  4. Arduino Ethernet Shield W5100 ($40.00)
  5. Arduino Ethernet Shield PoE module Ag9120-S ($21.00 from Amazon) (I moved the Arduino, and found I didn't need to use the switch for the Arduino, so save yourself $20 and buy yourself a new movie and put the Arduino in place with a power supply).
  6. SD card to hold the Arduino configuration ($4.00) (I ended up adding code to the Arduino that, if the SD card fails, will fall back to a coded configuration - and you can actually accomplish all of this without the SD card).
  7. Case for Arduino ($15-30, depending on what you want)
  8. Linux Server (this can range - I had one that had been decomissioned a while back)
  9. Ethernet cable, connectors, and jacks
We'll document the project in the following order :
  1. Arduino - Design and Code - should act as the interrupt-driven sensor interface with PoE so that it will report in when events happen in real-time
  2. Arduino Assembly and Testing - putting the active sensor together
  3. !!Bugs!! - found some gotchas, might help you to see these.
  4. Installing the Arduino and doing preliminary testing of at least two doors. This is pretty self explanatory - find a spot and a box, install the arduino to this, and run cables with the door sensors - examples of the installation can be found here).
  5. Configure the TrendNet TV-IP322P camera in a motion detection package. After failing with first ZoneMinder and then Linux Motion due to unacceptable frame rate recordings, I found I could use a  flood light connected to the alarm server documented in Arduino Assemly and Testing, but didn't want flood lights by every camera.  After testing and retesting, I found that the highest quality of uploaded video, the fastest to get set up, the easiest to tweak and reduce false positives was the camera's own motion detection and upload functionality.  Incidentally, it turns out that when the arduino was down, this still provided a layer of security as I could configure the CGI events to still alert me.
  6. Changing from a prototyping perf board to an actual PCB for a cleaner installation. The board looked great, connections and components fit (the relays were a little snug, but not too bad).  I did run into some issues with the early board (I deleted the old board and uploaded a new one to fix those problems), but things are looking up.  If you need a board professionally done and an initial small count for testing, BatchPCB is the way to go.  Testing the board was successful.
  7. Setting up a portal that includes current camera views, event logs (from sensors and camera recordings), and the ability to view live camera feeds.
  8. Full testing and results
Questions and comments are welcome.

Thursday, March 22, 2012

Corvette status update

Okay, it's been a while, and I thought I'd better post another update.  It's been slow work.  The carpet was a massive step toward completion, but now there are a few hang ups.  The hold ups I have :
Need thread repairs
  1. Thread repairs for drivers side quarter panel
  2. Sheet metal to cover the gaping hole where the transmission shift comes into the car
  3. Seat belts rebuilt (I am not rebuilding them myself, as some parts were broken and I cannot get a hold of them)
  4. Spring and retainer cup for the sunvisors.
  5. Design headlight actuators
  6. Paint headlight lids and bezels
  7. Get the stereo head unit, speakers/subs, and amps.
However, just to give you an idea of what else has been done, the window glass is in position (not in place as I need to connect the electrical to the window switches to move and adjust), the panels are painted and ready to install (except the drivers quarter panel), the battery has been purchased, the starter connected.  Once the battery is connected, I'll align and install the window glass, and then try to get the door panels in place.

    The panels have NEVER been in the car since I have owned it.  They were in boxes when I got the car, so this is THE most the car has been assembled for 15 years.  Thank goodness for the CAD drawings of the assembly - they identify what things I can install and how to do it, since I've never worked on some of these parts.


    And, despite the dust on the car, the windows being in position gives me a sense of having it almost ready.


    I'm getting a little antsy for this to be done!

    Tuesday, February 14, 2012

    Google Chrome, Linux (selinux), and Google Docs "Aw, Snap!"

    For people getting an "Aw, Snap!" when opening Google docs in Google's Chrome browser (OS is Linux, with selinux enabled, can be determined by running "getenforce"), you should be able to fix this by running the following :



    grep chrome /var/log/audit/audit.log|audit2allow -M chromepolicy

    This will greate a chromepolicy.pp file in the current directory.  It is a binary file, so don't panic if you less or vi the file and can't read it.  It basically spits what was being denied by selinux through audit2allow to create the policy.

    Once done, it prints this nice little line :

    [root@cyanide var]# grep chrome /var/log/audit/audit.log|audit2allow -M chromepolicy******************** IMPORTANT ***********************
    To make this policy package active, execute:

    semodule -i chromepolicy.pp

    [root@cyanide var]

    That "semodule -i chromepolicy.pp" line is the new key to making it work.  If you then run that, it will allow Chrome to do the things it's been denied.  This is a better option for those that want to keep selinux enabled.  (Disabling it is bad if you don't trust people around you.)

    Friday, February 10, 2012

    Arduino : Interrupt-Driven Alarm System

    Okay, I have now figured out how to do the interrupt-driven arduino alarm system, using a series of diodes for each input. I also found out that the resistors were pointless, as the inputs on the arduino typically contain a pull-up or pull-down resistor already embeded on the device. It makes it quite nice to set up an input.
    The code is available from  http://svn.silverhawk.net/files/interrupt_alarm.pde  , you just need an appropriate Arduino, the Ethernet Shield (with the SD card), a small prototyping board, wire, and some diodes.  It's configurable (on an SD card), and will connect to a server.  It uses encryption, so sniffing minds can't easily see what is being communicated (pretty effective PSK-based "algorithm" considering the low processing power of an Arduino - it's a simple PSK XOR mechanism).  If you need a schematic, let me know.  E-Mail address is in the code header.

    Lumberjack - An open source logging mechanism

    Hey;

    Just as an announcement, after watching some non-opensource log mechanisms, I thought I'd give a shot at creating my own.  The source is available for free, though the code isn't exactly "open source".  I'll do what I can to get patches committed into the application, for other releases.  If this becomes a stable and enterprise-ready platform, I might just release the control of it into the opensource world.

    You can obtain a copy of the source from :

    http://www.silverhawk.net/files/lumberjack-0.1.tgz

    Also, you can test it with the following instructions :


    1. First, compile, prepare, and launch :

      if [ -e Makefile ]; then make rmproper; fi && ./configure --modules-dir=modules && make && make modules && ./server/lumberjack -bd -f server/lumberjack.conf
    2. In another window, CD into clients, and compile the test client :

      make clean && make && ./followlog -v --no-daemon --verbose --type apache --path test/info.log --report-back plain://localhost:5820
    3. In a third window, add something to the log file :

      echo '['`date '+%a %b %d %H:%M:%S %Y'`']' - warning : work >> test/info.log

    Friday, January 27, 2012

    Okay, carpeting is finally installed.  Woohooo!  New car smell is back!  This is the most the car has been put together in 14 years!  Carpeting is installed.  Now I need some panels, and some speakers.  Some photos of the installed carpet :


    Note that the center console still hasn't been finished.  I have the wiring in position, but still need to cover the shifter (after converting from the TH350 automatic to a new Tremec TKO II standard).  I dropped a sheet over the passengers foot pan to protect the carpets for a little bit.


    And another photo of the rear compartment area, without the panels :


    Next up, is to pick up panels, finish the shifter console, get the rest of the wiring installed, install speakers, install a battery and then the door glass (and then the door panels), and the dash board.

    Then, headlights, plumb and lube the engine, and the suspension and steering joints and bearings, and then put the hood on.  Might get a little expensive (the only thing holding it back right now).  But, it's coming along!