Tag: zoneminder

openHAB – Motion Detection With Zoneminder Via SQL Triggers

We had used ZoneMinder filters to run a script which turned a “motion detected” switch on and off in openHAB. We had turned that off in favor of an openHAB/ZoneMinder binding, but polling ZoneMinder for motion events added a lot of load to the system. We tried re-enabling the filters we’d used previously, and they didn’t work. There are a lot of caveats around using filters (tl;dr: filtering can be delayed by several minutes, which renders ‘now’ filters ineffective) and more recent versions of ZoneMinder don’t have a number of alarm frames until after the event (which means filtering on alarm frames > 1 only detects motion after the fact). All of this means that the filters which worked pretty well a year or two ago no longer work reliably.

Architecturally, the ZoneMinder filter process seemed ill suited for our needs. Actions that are not time sensitive, like file cleanup or roll-up reporting, could be done through a filter. But it’s not a good solution for identifying the FexEx guy in the driveway. I decided to try a SQL TRIGGER (well, two triggers). Doing so required getting a UDF for sys_exec set up. To do so, I cloned lib_mysqludf_sys. The Makefile needs to be edited — LIBDIR needs to be updated with the appropriate directory for the MariaDB installation (/usr/lib64/mariadb/plugin/ in my case). I also needed to modify the install compile line to:

gcc -fPIC -Wall -I/usr/include/mysql/server -I. -shared lib_mysqludf_sys.c -o $(LIBDIR)/lib_mysqludf_sys.so

Running install.sh then installed and registered the UDF in my MariaDB server. The CLI SQL client can be used to verify sys_exec is working:

MariaDB [zm]> SELECT sys_exec('cat /etc/fedora-release');
+-------------------------------------+
| sys_exec('cat /etc/fedora-release') |
+-------------------------------------+
| 0 |
+-------------------------------------+
1 row in set (0.012 sec)

And finally, we can create the triggers to detect motion:

-- SHOW TRIGGERS
-- DROP TRIGGER zm.TriggerMotionOnNewEvent;
DELIMITER @@

CREATE TRIGGER TriggerMotionOnNewEvent
AFTER INSERT ON `Events`
FOR EACH ROW
BEGIN

DECLARE strCommand CHAR(255);
DECLARE strCameraName CHAR(64);
DECLARE iCameraID INT(10);
DECLARE iResult INT(10);
-- variables for local openHAB REST API hostname and port
DECLARE strOpenHABHost CHAR(64);
SET strOpenHABHost='http://openhabhost.example.com:8080';


-- Translate ZoneMinder IP camera ID with openHAB item name
SET iCameraID = NEW.monitorID;
IF(iCameraID = 10) THEN
SET strCameraName='IPCam05_Alarm';
ELSEIF(iCameraID = 11) THEN
SET strCameraName='IPCam03_Alarm';
ELSEIF(iCameraID = 12) THEN
SET strCameraName='IPCam04_Alarm';
ELSEIF(iCameraID = 13) THEN
SET strCameraName='IPCam01_Alarm';
ELSEIF(iCameraID = 14) THEN
SET strCameraName='IPCam02_Alarm';
END IF;

SET strCommand=CONCAT('/usr/bin/curl ', '-s --connect-timeout 10 -m 10 -X PUT --header "Content-Type: text/plain" --header "Accept: application/json" -d "ON" "',strOpenHABHost,'/rest/items/',strCameraName,'/state"');
SET iResult = sys_exec(strCommand);
END;
@@
DELIMITER ;

And to clear motion when the event ends:

-- SHOW TRIGGERS
-- DROP TRIGGER zm.ClearMotionOnEventEnd;
DELIMITER @@

CREATE TRIGGER ClearMotionOnEventEnd
AFTER UPDATE ON `Events`
FOR EACH ROW
BEGIN

DECLARE strCommand CHAR(255);
DECLARE iResult int(10);
DECLARE strCameraName CHAR(25);
DECLARE iCameraID int(5);
-- variables for local openHAB REST API hostname and port
DECLARE strOpenHABHost CHAR(64);
SET strOpenHABHost='http://openhabhost.example.com:8080';

-- Translate ZoneMinder IP camera ID with openHAB item name
SET iCameraID = NEW.monitorID;
IF iCameraID = 10 THEN
SET strCameraName='IPCam05_Alarm';
ELSEIF iCameraID = 11 THEN
SET strCameraName='IPCam03_Alarm';
ELSEIF iCameraID = 12 THEN
SET strCameraName='IPCam04_Alarm';
ELSEIF iCameraID = 13 THEN
SET strCameraName='IPCam01_Alarm';
ELSEIF iCameraID = 14 THEN
SET strCameraName='IPCam02_Alarm';
END IF;

IF NEW.EndTime IS NOT NULL THEN
SET strCommand=CONCAT('/usr/bin/curl ', '-s --connect-timeout 10 -m 10 -X PUT --header "Content-Type: text/plain" --header "Accept: application/json" -d "OFF" "http://',strOpenHABHost,':',iOpenHABPort,'/rest/items/',strCameraName,'/state"');
SET iResult = sys_exec(strCommand);
END IF;

END;
@@
DELIMITER ;

Now when new motion detection events are inserted into the Events database table, the openHAB item corresponding to the camera will be turned on. When the event record is updated with an end timestamp, the openHAB item corresponding to the camera will be turned off.

Using ZoneMinder v1.32.3 With OpenHAB2

I documented a temporary fix to return ZM_PATH_ZMS and ZM_OPT_FRAME_SERVER through the ./api/configs/view/<KEYNAME>.json API so ZoneMinder 1.31.45 worked with the OpenHAB2 binding. Upon upgrading ZoneMinder to 1.32.3, the binding was no longer able to communicate with our ZoneMinder server.

In the OpenHAB2 log, errors indicated malformed JSON was received.

Caused by: com.google.gson.stream.MalformedJsonException: Use JsonReader.setLenient(true) to accept malformed JSON at line 1 column 7 path $
at com.google.gson.stream.JsonReader.syntaxError(JsonReader.java:1568) ~[?:?]
at com.google.gson.stream.JsonReader.checkLenient(JsonReader.java:1409) ~[?:?]
at com.google.gson.stream.JsonReader.doPeek(JsonReader.java:542) ~[?:?]
at com.google.gson.stream.JsonReader.peek(JsonReader.java:425) ~[?:?]
at com.google.gson.JsonParser.parse(JsonParser.java:60) ~[?:?]
at com.google.gson.JsonParser.parse(JsonParser.java:45) ~[?:?]
at name.eskildsen.zoneminder.jetty.JettyConnectionInfo.fetchDataAsJson(JettyConnectionInfo.java:352) ~[?:?]

Using a web browser to access <ZoneMinderURL>/zm/api/configs/view/ZM_PATH_ZMS.json, malformed JSON is returned.

Conf files are not updated when new packages are installed – an conf.rpmnew is created instead. The changes from the new config (zoneminder.conf.rpmnew) file need to be merged into the existing config file (zoneminder.conf). In our zm.conf file, I added:

ZM_DB_SSL_CA_CERT=
ZM_DB_SSL_CLIENT_KEY=
ZM_DB_SSL_CLIENT_CERT=

Reloading the page in my browser confirmed that the JSON response is valid.

When the ZoneMinder binding started, it successfully attached to our monitors and detected a motion alarm.

This does not negate the need for the original fix — config.php still needs to have the strcmp I added. When ZoneMinder is upgraded, /usr/bin/zmupdate.pl is run (I needed to run “/usr/bin/zmupdate.pl -f” to stop zmc from existing with return code 255), the values I added to the ZoneMinder Config table are removed — they need to be re-added.

 

Temporary Fix: ZoneMinder, PHP7.2, openHAB ZoneMinder Binding

I got Zoneminder 1.31.45 (which includes the new CakePHP framework that doesn’t use what have become reserved words in PHP7) working with the openHAB ZoneMinder binding (which relies on data from the API at  /zm/api/configs/view/ATTR_NAME.json). There are two options, ZM_PATH_ZMS and ZM_OPT_FRAME_SERVER which now return bad parameter errors when attempting to retrieve the config using /view/. Looking through the database update scripts, it appears both of these parameters were removed at ZoneMinder 1.31.1

ZM_PATH_ZMS was removed from the Config database and placed in a config file, /etc/zm/conf.d/01-system-paths.conf. There is a PR to “munge” this value into the API so /viewByName returns its value … but that doesn’t expose it through /view.

ZM_OPT_FRAME_SERVER appears to have been eliminated as a configuration option.

You cannot simply re-insert the config options into the database, as ZoneMinder itself loads the ZM_PATH_ZMS value from the config file and then proceeds to use it. When it attempts to load config parameters from the Config table and encounters a duplicate … it falls over. We were unable to view our video through the ZoneMinder server.

*But* editing /usr/share/zoneminder/www/includes/config.php (exact path may vary, list the files from your package install and find the config.php in www/includes) to include an if clause around the section that loads config parameters from the database, and only loading the parameter when the Name is not ZM_PATH_ZMS (bit in yellow below) avoids this overlapping config value.

$result = $dbConn->query( 'select * from Config order by Id asc' );
if ( !$result )
   echo mysql_error();
   $monitors = array();
   while( $row = dbFetchNext( $result ) ) {
      if ( $defineConsts )
      // LJR 2018-08-18 I inserted this config parameter into DB to get OH2-ZM running, and need to ignore it in the ZM web code
      if( strcmp($row['Name'],'ZM_PATH_ZMS') != 0){
         define( $row['Name'], $row['Value'] );
      }
   $config[$row['Name']] = $row;
   if ( !($configCat = &$configCats[$row['Category']]) ) {
      $configCats[$row['Category']] = array();
      $configCat = &$configCats[$row['Category']];
   }
   $configCat[$row['Name']] = $row;
}

Once the ZoneMinder web site happily ignores the presence of ZM_PATH_ZMS from the database config table, you can insert it and ZM_OPT_FRAME_SERVER (an option which appears to have been removed at ZoneMinder 1.31.1) back into the Config table. **Important** — change the actual value of ZM_PATH_ZMS to whatever is appropriate for your installation. In my ZoneMinder installation, /cgi-bin-zm is the cgi-bin directory, and /cgi-bin-zm/nph-zms is the ZMS binary.

From a MySQL command line:

use zm; #Assuming your zoneminder database is actually named zm
INSERT INTO `Config` VALUES (225,'ZM_PATH_ZMS','/cgi-bin-zm/nph-zms','string','/cgi-bin-zm/nph-zms','relative/path/to/somewhere','(?^:^((?:[^/].*)?)/?$)',' $1 ','Web path to zms streaming server',' The ZoneMinder streaming server is required to send streamed images to your browser. It will be installed into the cgi-bin path given at configuration time. This option determines what the web path to the server is rather than the local path on your machine. Ordinarily the streaming server runs in parser-header mode however if you experience problems with streaming you can change this to non-parsed-header (nph) mode by changing \'zms\' to \'nph-zms\'. ','hidden',0,NULL);
INSERT INTO `Config` VALUES (226,'ZM_OPT_FRAME_SERVER','0','boolean','no','yes|no','(?^i:^([yn]))',' ($1 =~ /^y/) ? \"yes\" : \"no\" ','Should analysis farm out the writing of images to disk',' In some circumstances it is possible for a slow disk to take so long writing images to disk that it causes the analysis daemon to fall behind especially during high frame rate events. Setting this option to yes enables a frame server daemon (zmf) which will be sent the images from the analysis daemon and will do the actual writing of images itself freeing up the analysis daemon to get on with other things. Should this transmission fail or other permanent or transient error occur, this function will fall back to the analysis daemon. ','system',0,NULL);

Now restart ZoneMinder and the OH2 ZoneMinder binding. We’ve got monitors on the ZoneMinder web site, we are able to view the video stream, and OH2 picks up alarms from the ZoneMinder server.

If you re-run zmupdate.pl, it will remove these two records from the Config table. If you upgrade ZoneMinder, the change to the PHP file will be reverted.

Zoneminder Snapshot With openHAB Binding

When we upgraded to Fedora 28 on our server, ZoneMinder ceased working because some CakePHP function names could no longer be used. To resolve the issue, I ended up running a snapshot build of ZoneMinder that included a newer build of CakePHP. Version 1.31.45 instead of 1.30.4-7 on the repository.

All of our cameras showed up, and although the ZoneMinder folks seem to have a bug in their SQL query when building out the table of event counts on the main page (that is, all of my monitors have blank instead of event counts and my apache log is filled with

[Wed Aug 15 12:08:37.152933 2018] [php7:notice] [pid 32496] [client 10.5.5.234:14705] ERR [SQL-ERR 'SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'and E.MonitorId = '13' ),1,NULL)) as EventCount1, count(if(1 and (  and E.Monito' at line 1', statement was 'select count(if(1 and ( E.MonitorId = '13' ),1,NULL)) as EventCount0, count(if(1 and (  and E.MonitorId = '13' ),1,NULL)) as EventCount1, count(if(1 and (  and E.MonitorId = '13' ),1,NULL)) as EventCount2, count(if(1 and (  and E.MonitorId = '13' ),1,NULL)) as EventCount3, count(if(1 and (  and E.MonitorId = '13' ),1,NULL)) as EventCount4, count(if(1 and (  and E.MonitorId = '13' ),1,NULL)) as EventCount5 from Events as E where MonitorId = ?' params:13]

… it works.

Until Scott checked openHAB, where all of the items are offline. Apparently the openHAB ZoneMinder binding is using the cgi-bin stuff to get the value of ZM_PATH_ZMS. A config option which was removed from the database as part of the upgrade process.

Upgrading database to version 1.31.1
Loading config from DBNo option 'ZM_DIR_EVENTS' found, removing.
No option 'ZM_DIR_IMAGES' found, removing.
No option 'ZM_DIR_SOUNDS' found, removing.
No option 'ZM_FRAME_SOCKET_SIZE' found, removing.
No option 'ZM_OPT_FRAME_SERVER' found, removing.
No option 'ZM_PATH_ARP' found, removing.
No option 'ZM_PATH_LOGS' found, removing.
No option 'ZM_PATH_MAP' found, removing.
No option 'ZM_PATH_SOCKS' found, removing.
No option 'ZM_PATH_SWAP' found, removing.
No option 'ZM_PATH_ZMS' found, removing.
 207 entries
Saving config to DB 207 entries
Upgrading DB to 1.30.4 from 1.30.3

The calls from openHAB yield 404 errors in the access_log

10.0.0.5 - - [15/Aug/2018:09:38:04 -0400] "GET /zm/api/configs/view/ZM_PATH_ZMS.json HTTP/1.1" 404 1751 "-" "Jetty/9.3.21.v20170918"

 

Unfortunately they’ve changed the URL to get these values — it’s “munged” from the config file as the parameters are no longer stored to the Config table.
http://zoneminder.domain.ccTLD/zm/api/configs/view/ZM_PATH_ZMS.json
is now
http://zoneminder.domain.ccTLD/zm/api/configs/viewByName/ZM_PATH_ZMS.json

So … that’s a problem!

Zoneminder and PHP 7.2

After updating to php 7.2, ZoneMinder completely stopped working. Fortunately there were lots of entries in the error_log file

[Fri Aug 10 15:44:19.880809 2018] [php7:error] [pid 5293] [client 127.0.0.1:46958] PHP Fatal error:  Cannot use 'Object' as class name as it is reserved in /usr/share/zoneminder/www        /api/lib/Cake/Core/Object.php on line 30
[Fri Aug 10 15:44:19.889737 2018] [php7:warn] [pid 5293] [client 127.0.0.1:46958] PHP Warning:  file_put_contents(/var/lib/zoneminder/templogs/cake_error.log) [function.file-put-contents]: failed to open stream: No such file or directory in /usr/share/zoneminder/www/api/lib/Cake/Log/Engine/FileLog.php on         line 142
[Fri Aug 10 15:44:19.889850 2018] [php7:warn] [pid 5293] [client 127.0.0.1:46958] PHP Warning:  file_put_contents(/var/log/zonemindererror.log) [function.file-put-contents]: failed to open stream: Permission denied in /usr/share/zoneminder/www/api/lib/Cake/Log/Engine/FileLog.php on line 142
[Fri Aug 10 15:44:19.890176 2018] [php7:warn] [pid 5293] [client 127.0.0.1:46958] PHP Warning:  file_put_contents(/var/lib/zoneminder/templogs/cake_error.log) [function.file-put-contents]: failed to open stream: No such file or directory in /usr/share/zoneminder/www/api/lib/Cake/Log/Engine/FileLog.php on         line 142
[Fri Aug 10 15:44:19.890221 2018] [php7:warn] [pid 5293] [client 127.0.0.1:46958] PHP Warning:  file_put_contents(/var/log/zonemindererror.log) [function.file-put-contents]: failed to open stream: Permission denied in /usr/share/zoneminder/www/api/lib/Cake/Log/Engine/FileLog.php on line 142
[Fri Aug 10 15:44:19.890594 2018] [php7:warn] [pid 5293] [client 127.0.0.1:46958] PHP Warning:  file_put_contents(/var/lib/zoneminder/templogs/cake_error.log) [function.file-put-contents]: failed to open stream: No such file or directory in /usr/share/zoneminder/www/api/lib/Cake/Log/Engine/FileLog.php on         line 142
[Fri Aug 10 15:44:19.890637 2018] [php7:warn] [pid 5293] [client 127.0.0.1:46958] PHP Warning:  file_put_contents(/var/log/zonemindererror.log) [function.file-put-contents]: failed to open stream: Permission denied in /usr/share/zoneminder/www/api/lib/Cake/Log/Engine/FileLog.php on line 142
[Fri Aug 10 15:44:19.890960 2018] [php7:warn] [pid 5293] [client 127.0.0.1:46958] PHP Warning:  file_put_contents(/var/lib/zoneminder/templogs/cake_error.log) [function.file-put-contents]: failed to open stream: No such file or directory in /usr/share/zoneminder/www/api/lib/Cake/Log/Engine/FileLog.php on         line 142
[Fri Aug 10 15:44:19.891021 2018] [php7:warn] [pid 5293] [client 127.0.0.1:46958] PHP Warning:  file_put_contents(/var/log/zonemindererror.log) [function.file-put-contents]: failed to open stream: Permission denied in /usr/share/zoneminder/www/api/lib/Cake/Log/Engine/FileLog.php on line 142
[Fri Aug 10 15:44:19.892818 2018] [php7:error] [pid 5293] [client 127.0.0.1:46958] PHP Fatal error:  Uncaught Error: Class 'Controller' not found in /usr/share/zoneminder/www/api/li        b/Cake/Error/ExceptionRenderer.php:174\nStack trace:\n#0 /usr/share/zoneminder/www/api/lib/Cake/Error/ExceptionRenderer.php(92): ExceptionRenderer->_getController(Object(InternalErr        orException))\n#1 /usr/share/zoneminder/www/api/lib/Cake/Error/ErrorHandler.php(126): ExceptionRenderer->__construct(Object(InternalErrorException))\n#2 /usr/share/zoneminder/www/ap        i/lib/Cake/Error/ErrorHandler.php(284): ErrorHandler::handleException(Object(InternalErrorException))\n#3 /usr/share/zoneminder/www/api/lib/Cake/Error/ErrorHandler.php(213): ErrorHa        ndler::handleFatalError(64, 'Cannot use 'Obj...', '/usr/share/zone...', 30)\n#4 /usr/share/zoneminder/www/api/lib/Cake/Core/App.php(970): ErrorHandler::handleError(64, 'Cannot use '        Obj...', '/usr/share/zone...', 30, Array)\n#5 /usr/share/zoneminder/www/api/lib/Cake/Core/App.php(943): App::_checkFatalError()\n#6 [internal function]: App::shutdown()\n#7 {main}\n          thrown in /usr/share/zoneminder/www/api/lib/Cake/Error/ExceptionRenderer.php on line 174

Looks like CakePHP used class names that are now reserved words. Unfortunately, you cannot just drop the updated CakePHP files into ZoneMinder (I tried). Until the repository package is updated, you’ve got to build ZoneMinder from source. Or grab the testing RPM from the zoneminder.com repo. 1.31.45-1.13 works. Don’t forget to run “perl /usr/bin/zmupdate.pl” to update the database.

Then I had to throw the database connection into from the config file into /usr/share/zoneminder/www/api/app/Config/database.php (default array) because I do not use the default connection info.

Once I had the updated ZoneMinder along with the newer CakePHP that the ZM folks have in their repo … we’ve got ZoneMinder again.

ZoneMinder After Upgrade

We recently updated from ZoneMinder 1.30 to 1.34 – easy as can be, ran the DB update script and everything came right online. Except … our home automation system hasn’t been able to access the system. OpenHAB reports that the bridge is offline. And we’re getting 404 errors in all of the /zm/api calls in access_log.

Turns out the API was offline because when the new package came down … there was a zoneminder.conf.rpmnew in the Apache conf.d directory. Can’t even say I found this intentionally – I wanted to check the Apache config file to see if it had anything about the api directory, did a directory listing, and said oooooh!

[lisa@fedora01 conf.d]# ll zone*
-rw-r–r– 1 root root 1990 Jul 29 18:13 zoneminder.conf
-rw-r–r– 1 root root 1990 Aug 28 22:34 zoneminder.conf.rpmnew

They’ve changed a few of the sub-directories and added components to the config. As soon as I renamed zoneminder.conf to zoneminder.conf.old, copied zoneminder.conf.rpmnew to zoneminder.conf, and repeated a few config tweaks we had made for the original installation … restarted Apache and voila, we can fetch /zm/api/host/getVersion.json and get values. So if you’re getting odd 404 errors and CakePHP “/zm/api” not found errors maybe you forgot to update your config with changes from the rpmnew file.

Zoneminder Setup

I just installed ZoneMinder tonight. I don’t know if I missed a section in the documentation or something’s just missing — there’s doc for getting the build onto your box (or building your own from source). I didn’t want to install the package and all of its dependencies from their repo, so I manually installed the prereqs from the standard Fedora repositories. Installed the zoneminder rpm and switched from the ‘installation guide’ to the ‘user guide’. Which starts out on a web site. Umm … what web site? There’s definitely something missing here. Their package drops a config file in /etc/httpd/conf.d … *but* it presupposes that it is a dedicated server (which, in fairness, is the recommended configuration).

I edited the zoneminder.conf file and threw the whole thing in a VirtualHost tag, added an SSL cert for the hostname I’m using, and restarted Apache. OK, that’s better … I get *something*. Unfortunately ‘something’ is a massive MySQL error.

Some searching (rpm -ql zoneminder, then search through the files the package installed for something that looks good) yielded a config file at /etc/zm/zm.conf. Went in there, defined a database, user, and password. Created said database & user in mysql & assigned rights. Umm, stranger database error that leads me to believe the database hasn’t been initialized. Oops. LMGTFY. Found a SQL file at /usr/share/zoneminder/db/zm_create.sql and imported *that* … wasn’t sure if it was as simple as changing the zm database name in the SQL file to what I wanted or not, so I just used their default database name. Went back into MySQL and assigned the user I’d created earlier rights to the zm database and dropped *my* database. (Note, if you don’t use PHP date ‘stuff’, you may need to define the time zone in your php.ini file … I’d already set a default for other purposes).

Refresh and voila, I’ve got a system. You’ve got to click ‘start’ at the top for it to, well, start pulling in video feeds. I set up a VERY basic link to our WansView camera — ffmpg type, source is the full videostream.cgi URL, and remote method is “RTP/Unicast”. I can now see the video from our camera via the web site. WooHoo!

Now we need to get the motion detection set up & play with all of the options 🙂