Category Archives: Other

An OpenSprinkler success story

I wanted to automate the watering system at home preferably using open-source and DIY systems. The initial plan was to go with plain RPi, OpenHAB and some GPIO code driving the sprinkler valves but the problem was creating a useful UI to control the system since OpenHAB is too clunky and generic looking. I was also not quite ready diving deep into embedded programming and OpenHAB programming model. OpenSprinkler  seemed to have everything I needed, a RPi hat with all the correct electrical outputs and an open source firmware and android app I could modify myself if needed. In the end, programming the sequences myself and trying to make a decent UI would be just too much work for a small pet project so I went with a ready solution.

The requirements

  1. Three separate zones around the house, max 7 sprinklers per zone.
  2. Each zone must be turned on separately due to the pressure requirement for the sprinklers to work.
  3. Pump that drives the water must be turned on automatically with each zone valve.

Setting up the OSPi

OpenSprinkler offers fully assembled systems but I decided to go the DYI route using my own RPi and just buying the OSPi hat.

  • RPi 4 Model B 2GB
  • RPi official charger
  • OSPi (VAC)
  • 32GB SD card
  • Orbit 57056 2-Pin European Transformer

Finding a 24VAC power supply with EU plug was quite a challenge, the listed model from Orbit was one of the rare ones I could find online (on Amazon).

It is mentioned in OpenSprinkler documentation that a separate power supply for RPi is recommended. This was confirmed while testing where I saw dmesg errors about voltage not being sufficient and RPi rebooting endlessly. I ended up using the official RPi charger and the 24VAC charger at the same time.

Installing raspbian and OSPi firmware was easy with no problems encountered. Assembling the OSPi was also not problematic, other than drilling some holes into the supplied enclosure for the USB cable and WiFi adapter.

The WiFi

The built in WiFi on RPi would not work even half of the required distance and was simply horrendous. Onboard WiFi can be disabled by modifying /boot/config.txt and adding

dtoverlay=disable-wifi
in the [all] section.

After checking compatibility lists and reviews for RPi compatible USB WiFi adapters I went with Edimax EW-7811UN. I disabled the integrated card and configured the /boot/wpa_supplicant.conf to connect to the dedicated WiFi extender AP as a priority.

In the end I managed to achieve a not so great but stable signal from the house to controller box at around -60db. For the WiFi extender I went super cheap using TP-LINK TL-WR840N (15EUR) and positioning it with no walls blocking the signal other than a single garage door. I also added a small script to sudo cron to automatically restart RPi in case of any network downtime.

Driving the Pump

Looking for a relay to turn the pump on and off I decided to go with an off delay relay as an extra safety that will automatically turn off after the selected period of time. This is just in case OSPi goes haywire and does not turn off as scheduled or someone makes a mistake of turning the sprinklers on for too long. Pump draining all the water and running dry is a very bad scenario I would like to avoid. The model is a Tracon multifunction relay AC/DC 12-240V driven by 24VAC OSPi.

Putting it together

Relay is connected to OSPi port 0 (master zone) which is always turned on with either valve 1, 2 or 3. Relay drives the first power socket for the pump. The other two sockets are for Orbit 24VAC and RPi charger. This way the pump can be disconnected at any time and used manually.

The valves

24VAC valves are quite common. I found three candidates from Orbit, Rainbird and Cleber. In the end it came down to price and availability, so I went with 3x Rainbird CP075 off eBay, roughly 30$ each.

Finally, to connect the valves to OSPi I got some 4×0.75mm cable and some waterproof clips to connect them on the valve side. These are automatic clips put in a box full of gel which seals it when closed.

Operation and conclusion

It turns out the OSPi firmware and app has the exact functions I need to drive the setup. Master zone translates perfectly into the pump relay. For each valve, the "continuous" setting (which is default) allows you to setup a single schedule program and OSPi will automatically drive each valve one after another and not all at once (which would not work due to low pressure). Without the continuous setting one would have to write a separate program for each valve which is a bit clunky.

One thing that does not work quite as good is automatic rain delay. The idea is, if sprinklers are scheduled to work today but there is a rain forecast, delay the program for some time, like a day. Unfortunately, if it does not rain at all, delay is still present. It would appear that OSPi only checks the forecasts but does not adjust the delay according to actual mm of rain that has fallen. I need to research this function in more depth to figure out the exact behavior and whether I can improve upon it.

Another glitch that appears once a month or so is that OSPi is randomly not accessible. This is fixed with the main router reboot. I am not sure yet what exactly causes the problem, whether the auto-reboot script works.. more investigation is needed. It probably boils down to not so great WiFi connection.

In the end I am quite happy I went with the OpenSprinkler and not a full DIY solution. It saved time, does everything I require and I am able to modify it if ever needed.

 

2023 update

After the system being dormant through winter of 2022/2023, the RPi would no longer boot. The Sandisk Ultra SD card seems to have got corrupted for some reason so I had to re-image and reconfigure OSPI again. I replaced the card with a Sandisk Industrial series card which are supposedly more tolerant to heat and cold. We'll see how long this one lasts.

Other than that the system is still working great.

Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

Apache http to https redirect – use 307

Who knew that a simple thing like HTTP redirects would be so complicated? It turns out clients will just change POST to GET on 301 (Postman, curl, everyone?), same with 302 which really behaves like 303 and that is also an old implementation "bug". Yeah, seriously.

If you have a REST API with POST (or other non-GET) request endpoints (who doesn't?) this behaviour will completely destroy everything.  Many guides (top google results) out there for configuring Apache redirect do not mention this problem. The code of choice would be 308 Permanent Redirect but that is fairly new so I would not risk it, older clients don't know it exists. The only thing left is 307 which does not allow changing methods on redirect – exactly how it should be.

Solution:

 

Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

resize Fedora root partition

Default root partition size on my Fedora installs usually becomes too small down the line to the point I can no longer install packages or perform the upgrades without removing packages or clearing dnf cache.

Therefore I wanted to shrink my home partition and add that space to root.

We can't perform the resize while partitions are mounted so we need to boot in emergency or rescue mode. I first tried the emergency mode but the boot would lock up at Fedora logo so I decided to go with rescue instead.

Once in grub menu, press e to edit. At the end of the line of linux16 or linuxefi entry, add

Press Ctrl+x to boot with modified parameters. Once in rescue mode, perform the resize:


Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

Tenant resource authorization in JAX-RS

You have a book REST resource and each book has an owner. Only the owner of the book can access an owned book. JAX-RS specification has no answer to this problem since it only provides a role based security with @RolesAllowed annotation. It is unfortunate JavaEE spec does not offer at last some interfaces which we could then imlement for this purpose.. we need to roll our own. There are many ways this can be achieved, I will present one way of doing it.

Owned JPA entities extend a common class

All owned entities should extend a common class, let's call it OwnedEntity.

Protect owned resources with an interceptor

Create an interceptor which we will use on each owned resource that will check the owner of the entity against the authorized user. We pass the owned entity as a parameter. We will need this information to be able to fetch the correct JPA entity in the interceptor implementation.

We protect an owned resource with this interceptor

Interceptor implementation

Make sure the priority of this interceptor is lower than your security interceptor, since a valid authenticated user should already be present before it.

The limitation of this interceptor is that it can only protect ID based resources of type /resource/:id. For list resources, use seperate logic to insert an additional WHERE filter by owner ID to TypedQuery/Criteria query used for list fetching.

Second limitation is that the entity ID should always be declared first in resource method. Another way would be to enforce the name "id" as the parameter name representing the entity ID, but this requires additional reflection info to get method parameter names.

The example here uses SecurityContext to retreive the authorized user. You might need to inject your own context or parsed JWT token to retreive the needed identificator, depending on what you store in your database as owner ID (user UUID, email etc).

An improvement of this interceptor is to check the roles in security context and skip the owner check if role is an ADMIN or similar, since we probably want to allow admins to access all resources.

So how useful is this?

Good:

+protects owned resources with a simple annotation

Not so good:

-only protects ID based resources, you still need a seperate mechanism for lists
-only protects the base entity, not nested owned relations (/book/:id/somethingElse/:id2), which would mean child entity can have different owner than parent and client must be prevented from access of the child. I did not yet stumble upon such a requirement though.
-forcing method parameter position or consistent naming in resource methods Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

Creating a new torrent and seeding with Transmission

You have setup a Transmission server on your Linux box together with Transmisison Web or something along those lines and now you are wondering.. how can I actually seed a NEW file?

I couldn't find a straightforward answer on the web so here is the short tutorial:

  1. Upload your file to your transmission download directory
  2. cd to that directory and create a torrent file (lets say the file you uploaded was called  myfile.rar):

    Replace tracker1, tracker3, tracker3, …trackerN with a bunch of trackers. Better specify more than one in case they go down. Here is a cool little list of public trackers.
  3. Download the new .torrent file you just created, open Transmission Web and add the torrent. Since the file already exists in download directory, Transmisison will just revalidate the data and start seeding. *mind blown*
  4. Distribute the torrent file to your people or generate a magnet link with

     

Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

Watching beIN SPORTS Direct Spain from anywhere in the world

It's 2017 and it is almost impossible for a regular human being to watch Champions League without being subscribed to cable or internet TV and pay exorbitant fees for hundreds of bundled channels you don't really need or want.

beIN SPORTS Direct is 9.99€ a month in Spain which is a reasonable price. The problem is that it is geoblocked to Spain only. Here is a guide on how to overcome this.

  1. Register account

    Go to https://tiendas.beinconnect.es/contratar/promo/bein and insert your email and password. This will open up the following form:

    Nombre: your name
    Apellidos: surname
    Tipo documento: NIF
    NIF number: NIF is some kind of tax ID in Spain. It consists of 8 digits plus CRC character on the end. beIN sports has a validator on this number so it must be correct. Go to http://www.cespedes.org/dni2nif/ and insert random 8 digits, then copy the digits and the CRC.
    Teléfono móvil: This has to be a valid Spanish phone number. it starts with a 6 or 7 followed by 8 digits. You can put whatever valid number you like, there is no SMS confirmation or anything like that.
    Código postal: valid Spanish postal code. Pick whatever: https://en.wikipedia.org/wiki/List_of_postal_codes_in_Spain

    Finish up by giving your credit card info.

  2. VPN

    You should now have an account but if you try to play any of the streams they will be black unless you already have a Spanish IP. All you need to do now is find a decently priced VPN that allows you to freely switch locations and has servers in Spain. For a quick test you can do it via TunneloVPN Chrome xtension https://chrome.google.com/webstore/detail/tunnello-vpn-unblock-ultr/hoapmlpnmpaehilehggglehfdlnoegck which offers 1GB for free upon registration. In the extension choose Spain from the drop-down, turn it on and refresh your beIN SPORTS Direct page. Video should now start playing. You can verify that you have Spanish IP by going to any of the "what is my ip" websites.

And that's it, you can now watch beIN SPORTS Direct for 9.99€ a month from any country. You'll probably have to add 5€ to that monthly bill for a VPN but such is life. This is the cheapest way for a cord cutter to watch Champions League and La Liga online, at least known to me.

The only bad news is no HD streams which sucks.. but we'll get to this in a few years hopefully. Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

Oh FileZilla…

I have encountered a weird problem when connecting to our FreeBSD server with FileZilla over SFTP. Either with password or key authentication I would get:

Error:    Server sent disconnect message
Error:    type 2 (protocol error):
Error:    "Too many authentication failures"

So let's turn on debugging shall we?

 

Trace:    Pageant is running. Requesting keys.
Trace:    Pageant has 15 SSH-2 keys
Trace:    Successfully loaded 1 key pair from file
Trace:    Trying Pageant key #0
Trace:    Server refused our key
Trace:    Trying Pageant key #1
Trace:    Server refused our key
Trace:    Trying Pageant key #2
Trace:    Server refused our key
Trace:    Trying Pageant key #3
Trace:    Server refused our key
Trace:    Trying Pageant key #4
Trace:    Server refused our key
Trace:    Trying Pageant key #5
Trace:    Received disconnect message (protocol error)
Trace:    Disconnection message text: Too many authentication failures

So basically, I give Filezilla a specific keyfile but it tries all my keys anyway. Now let's see what the bright minds on FileZilla issue tracker have to say about this bug.

https://trac.filezilla-project.org/ticket/7739 gives us a workaround:

which works nicely. A working workaround is a blessing if you really need to use someting that is essentially broken. The bug is marked as a duplicate of https://trac.filezilla-project.org/ticket/5480

This bug contains a brilliant comment by an apparent FileZilla developer:

This is by design, FileZilla uses the system's SSH agent.

Just reconfigure the server to allow for more keys.

What the actual? The bug will apparently be solved via https://trac.filezilla-project.org/ticket/8232

which is marked as "fixed" and the comment 19 months ago says it will be in the "next version". The latest version is 3.24.0 released on January 1st 2017 which is exactly what I have and guess what? Not fixed, after 7 years.

 

So at this point I'll just safely assume that FileZilla might as well be the worst SFTP client in existence and just use something else. But guess what? There is more. The exact same problem exists in Gnome Files if you try to open an sftp:// location. The obvious reason is that Gnome Files does not ask you anything about keys or athenticaton type but just cycles through SSH keys to try and find the correct one. Why did nobody think about offering me a popup dialog to pick the correct key? Probably because Gnome likes to dumb down things, I can't really find any other reason.

Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

Expose your dev machine to the public via reverse SSH tunnel

Scenario: you are creating a REST service which needs to be exposed to the public even in early stage of development due to an upstream provider which sends back feedback data from a webhook API.

You are also behind a NAT so you'd have to port forward yourself out but you can't do that for whatever reason. Or maybe you are behind a firewall and 7 proxies.

All you need is an external server with a public IP. Then, on your machine:

ssh -R 0.0.0.0:8080:localhost:8080 server_user@public_server_ip

In the above example, I used port 8080 which my REST service uses when I develop. On your public server, make sure you have

GatewayPorts yes

in /etc/ssh/sshd_config. If it is missing, add it.

And that's it.. your local REST service is now publicly accessible via public_server_ip:8080.

a-m-a-z-i-n-g Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

Qt 5 + MSVC 2015 + OpenSSL + Windows XP Support = absolute mess

Recently I decided to move all of our C++/Qt based projects up to the latest version of Qt and C++ toolkits. That means Qt 5, Visual Studio 2015 and all the latest libraries which are needed either for Qt or standalone. There was one caveat tho.. we still need to support Windows XP because a large amount of our player base still uses it.  The nightmare begins.

This guide talks about:

-building shared release libraries of OpenSSL with msvc2015 XP target

-build shared release Qt framework with msvc2015 XP target

-build your application with msvc2015 XP target

1. Getting OpenSSL (1.0.1.p)

Building OpenSSL with MSVC 2015 and Windows XP target is an absolute clusterfuck. Forget XP target, even without that it simply WON'T compile with the usual instructions which used to work in the past (ex: http://developer.covenanteyes.com/building-openssl-for-visual-studio/). Thankfully, I found pre-built binaries and a useful batch script at http://www.npcglib.org/~stathis/blog/precompiled-openssl/. The pre-built binaries are ok but they are not built with XP target. They are also built with MT/MD naming which Qt does not really like (Qt insists that you link to libeay32 and ssleay32 and nothing else). Simply renaming them does not really work because .lib tries to include the wrong thing and you end up needing to provide both (no go).

So after a full day of working around the various problems I managed to compile OpenSSL with msvc2015 and XP target. This is a rough procedure so I might have missed something because I recompiled at least 100 times before it worked, but it should at least guide you on the right track:

  • Modify the batch script from the link above (you need Cygwin, perl.. ) and make it to a working state
  • Run these in cmd: http://doc.qt.io/qt-5/windows-issues.html (make sure you have Windows SDK installed and run the 2015 vcvarsall.bat version of the script, obviously)
  • Open ms/ntdll-x86.mk that is generated
  • Add the same env vars you just added in cmd to the top of makefile:
  • Add

    to CFLAGS
  • Add

    to CFLAGS
  • Replace /subsystem flag in LFLAGS with /subsystem:console,5.01
  • Also add the same subsystem flag to MKLIB and MKFLAGS
  • Remove MD from SSL and CRYPTO flags so you get libeay32 and libssl32 named binaries
  • In root OpenSSL folder run:

    This should produce XP targeted OpenSSL libraries. You know they are XP targeted if you open both DLLs in HEX editor and check rows 130-150, somewhere in those rows you need to se 05 00 01 … appear twice. See: http://www.tripleboot.org/?p=423

2. Build Qt with Windows XP support

Setup the environment again: http://doc.qt.io/qt-5/windows-issues.html

Then configure Qt, I personally used:

where -I and -L flags need to point to OpenSSL include files and the libraries you just built in previous step. We are doing a shared build (I used to like it all static but it turns out shipping Qt libraries bundled in the .exe with every single application update is a complete waste of bandwidth and time, at least 10MB every time).

Then build the Qt with: jom
Do not use nmake because jom is much faster!

After Qt builds, add it as a Kit in Qt Creator. You need to specify path to qmake.exe which is in qt-everywhere-opensource-src-5.X.X\qtbase\bin.

4. Configuring your Application Qmake

Now to your actual project. Add this to your qmake (.pro) file:

Second one? Because reasons: http://blogs.msdn.com/b/vcblog/archive/2009/08/27/windows-sdk-v7-0-v7-0a-incompatibility-workaround.aspx

5. Deployment

Add all Qt DLL libraries in the same folder as your .exe (from qtbase/lib), depending on the modules, it is usually at least:

QtCore.dll, QtGui.dll, QtNetwork.dll, QtWidgets.dll…

Also do not forget to include "platforms/qwindows.dll" in the same folder as your .exe. You also need to provide ssleay32.dll and libeay32.dll the same way. You might also need to provide msvc C++ redistributable dll files but I usually just instruct users to install the full C++ redistributable package from Microsoft.

 

And there you have it, Qt 5 with OpenSSL built with MSVC 2015 and with Windows XP support. An absolute mess if you are starting from zero and don't know all the little annoyances but once you do it's a relief. Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

Rode NT-USB Linux support

Googling around, I could not find a single mention whether Rode NT-USB microphone supports Linux or not. After some semi-encouraging words from Rode support I decided to go ahead and buy it anyway. I am happy to announce that it works out of the box and so far I did not encounter any problems on Fedora 23. The microphone is properly detected in Pavucontrol and Open Broadcaster. I am a complete amateur but to me the sound is crisp clear with zero background noise. The comparison with my old Siberia V2 headphone microphone or my in-built laptop one is night and day. These two would produce a lot of noise as soon as increasing the volume over 100% (at 100% they would be way too silent) and it was a complete disaster. NT-USB can go much louder and has no background noise even sitting 40cm from my loud laptop fan. Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs