Expose your dev machine to the public via reverse SSH tunnel

Scenario: you are creating a REST service which needs to be exposed to the public even in early stage of development due to an upstream provider which sends back feedback data from a webhook API.

You are also behind a NAT so you'd have to port forward yourself out but you can't do that for whatever reason. Or maybe you are behind a firewall and 7 proxies.

All you need is an external server with a public IP. Then, on your machine:

ssh -R 0.0.0.0:8080:localhost:8080 server_user@public_server_ip

In the above example, I used port 8080 which my REST service uses when I develop. On your public server, make sure you have

GatewayPorts yes

in /etc/ssh/sshd_config. If it is missing, add it.

And that's it.. your local REST service is now publicly accessible via public_server_ip:8080.

a-m-a-z-i-n-g

Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

OJDBC7 in a Docker container? Prepare for trouble

Scenario: A JDK8 Docker container using OJDBC7 to connect to the database. Sounds simple enough, what could go wrong?

Simptoms: Connecting to the database randomly takes several minutes, fails with a weird SqlRecoverableException: no more data to read from socket or just works fine as if there is no problem.

The same Docker image also works fine on some machine but fails consistently on other.

The reason is this. Docker is not good at /dev/random. Probably even more so if you run it in a VM, since it's double isolated from actual entropy sources (my non scientific observation). For whatever reason, OJDBC defaults to /dev/random and this causes a block when connecting to the database due to high probability of /dev/random depletion.

Simple solution is to just mount /dev/urandom to /dev/random inside the Docker, in docker run command:

-v /dev/urandom:/dev/random

So.. if you ever want to use OJDBC inside Docker, remember this flag. It will save lives or at least spare you hours of useless debugging.

 

Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

JPQL: getting whole entities on a distinct column

If you want distinct entities in JPQL you would normally write something like this:

SELECT DISTINCT(l) FROM Letter l WHERE l.name=:name;

But this will do a DISTINCT on the whole Entity. This will also fail on Oracle DB if your Entity contains a CLOB. What if you really want to do a DISTINCT on a field, for example:

SELECT DISTINCT(l.id) FROM Letter l WHERE l.name=:name;

Unfortunately, this only returns an array of ID fields. If you want to retrieve the full entities and do a DISTINCT on a field the final query looks like:

SELECT ll FROM Letter ll WHERE ll.id IN (SELECT DISTINCT(l.id) FROM Letter l WHERE l.name=:name);
Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

Fedora 24 XFCE pains and gains

I recently installed Fedora 24 XFCE on my brand new Entroware Apollo, a Linux friendly laptop. Here is a list of problems I encountered during my first week of use. While most of these problems had a solution with a bit of googling, non-tech savvy person would have severe problems solving them.

PAINS

  1. System completely hangs when I connect a second monitor. Unresolved.
  2. Could not see any WiFi networks because the interface was not managed. Had to make it managed in NetworkManager.conf manually.
  3. At some point, XFCE panel would not show up anymore and I got an error message on startup asking me to start the panel. Solved it by deleting .config folder. What the hell?
  4. No login prompt when laptop comes out of suspend, even though I have both option in power management and session settings turned on. Unresolved.
  5. Adding programs to favorites in Whiskers menu sometimes does not persist across reboots. I think esepcially when I did a hard reset due to issue #1.
  6. My Nexus 4 would not automount in Thunar over USB. After an hour of Googling and installing random packages I got it to work, I think?

GAINS

  1. Numix theme on XFCE is extremely nice, I love it. There is just one small bug so far in Volume widget, the selection color hides the slider completely.
  2. My FreeNAS was automatically detected by Thunar while the old Gnome-files setup would not show it and even refuse to mount it sometimes manually. A nice surprise.
  3. I like how you can fully customize XFCE panels. The only thing I actually couldn't do is completely hide the panel (there is like a 3px grey bar when the panel is hidden). Other than that it's super nice.
  4. Fedy is super nice. Props to it's maintainers, it all just works.

 

So basically, everything kinda works now but it was a bumpy road to achieve this. Hopefully I get some feedback on issue #1 in Fedora bug tracker because it's really severe.

Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

StormLib and BNCSUtil now available from repositories

Update 2023: repos are currently unavailable

 

Finally after years and years of screwing around with building StormLib or BNCSUtil everytime you moved your ghost bots to another server is finally over. As of today, I am hosting both libraries in apt and rpm repositories. In the process I also discovered that StormLib managed to get into Debian Testing (https://packages.debian.org/source/stretch/stormlib) so let's hope it'll be included in the next stable release!

I also  pushed CPack generator code into both repositories, so you can build these packages yourself. BNCSUtil was forked and cleaned especially.

DEB based distros

  1. To /etc/apt/sources.list add:
    #apt.xpam.pl
    deb https://apt.xpam.pl/ jessie main
  2. Add GPG key:
    wget -qO - https://apt.xpam.pl/xpam.pl-pubkey.asc | sudo apt-key add -
  3. Update and install:
    sudo apt-get update
    sudo apt-get install stormlib bncsutil

RPM based distros

  1. In /etc/yum.repos.d/rpm.xpam.pl.repo add
    [rpm.xpam.pl]
    name = rpm.xpam.pl
    baseurl = https://rpm.xpam.pl
    

    or, with dnf on Fedora:

    dnf config-manager --add-repo https://rpm.xpam.pl
  2. If using dnf, make sure to import the public key:
    rpm --import https://rpm.xpam.pl/rpm-pubkey.asc
  3. Update and install:
    yum update && yum install stormlib bncsutil

    or dnf equivalent.

 

This was a nice project to spend a few weekends on but there are more plans in 2016 to clean and tidy up the bnet tools we all use and love:
-add CMake and CPack support to our own bots, link against upstream StormLib and release a cleaned up ghost code, ease the setup process via repo installs with full dependency resolution (mysql, boost, storm and bncsutil). Provide precompiled binaries for Windows.
-provide patches to some other popular ghost forks to work with upstream StormLib.
-package PvPGN?
-provide pkgng packages for FreeBSD. This will probably involve writing pkgng generator for CMake's CPack which could be an interesting side project in itself.

Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

Qt 5 + MSVC 2015 + OpenSSL + Windows XP Support = absolute mess

Recently I decided to move all of our C++/Qt based projects up to the latest version of Qt and C++ toolkits. That means Qt 5, Visual Studio 2015 and all the latest libraries which are needed either for Qt or standalone. There was one caveat tho.. we still need to support Windows XP because a large amount of our player base still uses it.  The nightmare begins.

This guide talks about:

-building shared release libraries of OpenSSL with msvc2015 XP target

-build shared release Qt framework with msvc2015 XP target

-build your application with msvc2015 XP target

1. Getting OpenSSL (1.0.1.p)

Building OpenSSL with MSVC 2015 and Windows XP target is an absolute clusterfuck. Forget XP target, even without that it simply WON'T compile with the usual instructions which used to work in the past (ex: http://developer.covenanteyes.com/building-openssl-for-visual-studio/). Thankfully, I found pre-built binaries and a useful batch script at http://www.npcglib.org/~stathis/blog/precompiled-openssl/. The pre-built binaries are ok but they are not built with XP target. They are also built with MT/MD naming which Qt does not really like (Qt insists that you link to libeay32 and ssleay32 and nothing else). Simply renaming them does not really work because .lib tries to include the wrong thing and you end up needing to provide both (no go).

So after a full day of working around the various problems I managed to compile OpenSSL with msvc2015 and XP target. This is a rough procedure so I might have missed something because I recompiled at least 100 times before it worked, but it should at least guide you on the right track:

  • Modify the batch script from the link above (you need Cygwin, perl.. ) and make it to a working state
  • Run these in cmd: http://doc.qt.io/qt-5/windows-issues.html (make sure you have Windows SDK installed and run the 2015 vcvarsall.bat version of the script, obviously)
  • Open ms/ntdll-x86.mk that is generated
  • Add the same env vars you just added in cmd to the top of makefile:
  • PATH=C:\Program Files (x86)\Microsoft SDKs\Windows\v7.1A\Bin;$(PATH)
    INCLUDE=C:\Program Files (x86)\Microsoft SDKs\Windows\v7.1A\Include;$(INCLUDE)
    LIB=C:\Program Files (x86)\Microsoft SDKs\Windows\v7.1A\Include;$(LIB)
  • Add
    -D_USING_V110_SDK71_

    to CFLAGS

  • Add
    -I"C:\Program Files (x86)\Microsoft SDKs\Windows\v7.1A\Include" -I"C:\Program Files (x86)\Microsoft SDKs\Windows\v7.1A\Include"

    to CFLAGS

  • Replace /subsystem flag in LFLAGS with /subsystem:console,5.01
  • Also add the same subsystem flag to MKLIB and MKFLAGS
  • Remove MD from SSL and CRYPTO flags so you get libeay32 and libssl32 named binaries
  • In root OpenSSL folder run:
     nmake -f ms/ntdll-x86.mak

    This should produce XP targeted OpenSSL libraries. You know they are XP targeted if you open both DLLs in HEX editor and check rows 130-150, somewhere in those rows you need to se 05 00 01 … appear twice. See: http://www.tripleboot.org/?p=423

2. Build Qt with Windows XP support

Setup the environment again: http://doc.qt.io/qt-5/windows-issues.html

Then configure Qt, I personally used:

configure -target xp -release -qmake -opensource -nomake examples -opengl desktop -platform win32-msvc2015 -openssl -I C:\Users\me\git\openssl-1.0.1p-vs2015\include -L C:\Users\me\git\openssl-1.0.1p-vs2015\shared OPENSSL_LIBS="-llibeay32 -lssleay32 -lgdi32"

where -I and -L flags need to point to OpenSSL include files and the libraries you just built in previous step. We are doing a shared build (I used to like it all static but it turns out shipping Qt libraries bundled in the .exe with every single application update is a complete waste of bandwidth and time, at least 10MB every time).

Then build the Qt with: jom
Do not use nmake because jom is much faster!

After Qt builds, add it as a Kit in Qt Creator. You need to specify path to qmake.exe which is in qt-everywhere-opensource-src-5.X.X\qtbase\bin.

4. Configuring your Application Qmake

Now to your actual project. Add this to your qmake (.pro) file:

DEFINES += _ATL_XP_TARGETING
DEFINES += PSAPI_VERSION=1
QMAKE_LFLAGS_WINDOWS = /SUBSYSTEM:WINDOWS,5.01

Second one? Because reasons: http://blogs.msdn.com/b/vcblog/archive/2009/08/27/windows-sdk-v7-0-v7-0a-incompatibility-workaround.aspx

5. Deployment

Add all Qt DLL libraries in the same folder as your .exe (from qtbase/lib), depending on the modules, it is usually at least:

QtCore.dll, QtGui.dll, QtNetwork.dll, QtWidgets.dll…

Also do not forget to include "platforms/qwindows.dll" in the same folder as your .exe. You also need to provide ssleay32.dll and libeay32.dll the same way. You might also need to provide msvc C++ redistributable dll files but I usually just instruct users to install the full C++ redistributable package from Microsoft.

 

And there you have it, Qt 5 with OpenSSL built with MSVC 2015 and with Windows XP support. An absolute mess if you are starting from zero and don't know all the little annoyances but once you do it's a relief.

Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

Rode NT-USB Linux support

Googling around, I could not find a single mention whether Rode NT-USB microphone supports Linux or not. After some semi-encouraging words from Rode support I decided to go ahead and buy it anyway. I am happy to announce that it works out of the box and so far I did not encounter any problems on Fedora 23. The microphone is properly detected in Pavucontrol and Open Broadcaster. I am a complete amateur but to me the sound is crisp clear with zero background noise. The comparison with my old Siberia V2 headphone microphone or my in-built laptop one is night and day. These two would produce a lot of noise as soon as increasing the volume over 100% (at 100% they would be way too silent) and it was a complete disaster. NT-USB can go much louder and has no background noise even sitting 40cm from my loud laptop fan.

Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

The state of SVG rendering

This is an interesting little experiment I did when I was searching for a FreeBSD SVG logo to put on a poster. As you might know, SVG format is for vector images, allowing for infinite scaling without pixelation, which is exactly what you need, if you are going to print something on a big quality poster. Either that or you need originals in huge sizes for downscaling.

What caught my eye when I opened the image with Firefox was the lack of detail inside the devil ball which should be visible. This sent me on a little crusade to find out which browsers and image programs are actually capable of rendering it correctly.

For (assumingly) correct reference render we will take the logo from the freebsd.org:

Official FreeBSD logo as seen on freebsd.org

 

Now let's jump right into it..

FreeBSD logo rendered by Firefox
FreeBSD logo rendered by Firefox

Firefox render is very basic, lacks most of the highlights and internal details.

FreeBSD logo rendered by Chrome
FreeBSD logo rendered by Chrome

Chrome takes some artistic freedoms and completely screws up in the process. The elements are there but layered improperly plus the horns are now black. Obvious transparency and overlay issues.

FreeBSD logo rendered by IE
FreeBSD logo rendered by IE

Internet Explorer manages to produce the worst render of all programs.

FreeBSD logo rendered by Opera
FreeBSD logo rendered by Opera

Opera follows Chrome since they use the same underlying engine.

FreeBSD logo rendered by Safari
FreeBSD logo rendered by Safari

Safari also falls into the Webkit family with Chrome and Opera.

FreeBSD logo rendered by svg-edit
FreeBSD logo rendered by svg-edit

Svg-edit is an online JavaScript based tool.

FreeBSD logo rendered by Imagemagick
FreeBSD logo rendered by ImageMagick

ImageMagick is very close but it has an extra bright ring going through the middle which apparently shouldn't be there.

FreeBSD logo rendered by Inkscape
FreeBSD logo rendered by Inkscape

Inkscape is the only tool in this test that produced a proper render.

FreeBSD logo rendered by Gimp
FreeBSD logo rendered by Gimp

Gimp is also very close but has the same imperfection as ImageMagick.

FreeBSD logo rendered by IrfanView
FreeBSD logo rendered by IrfanView

IrfanView uses an external plugin to render SVGs that is not free. If we ignore the overlay text for a moment, the image suffers from Chrome-like problems except it's of absolute terrible quality.

 

And there you have it, the piss poor state of SVG rendering as of July 2015.

SVG protip

If you want to resize an SVG image and produce a high resolution PNG, the easiest method is to use ImageMagick from command line:

convert -density 600 Freebsd_logo.svg Freebsd_logo.png

Density is specified in dpi by default. You determine the dpi based on the size of the print you actually want to produce.

 

Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

Black screen with Catalyst > 13.4 on Asus N71jq and similar

If you own an Asus N71jq or a similar laptop model from 5 years ago you probably tried to update your Catalys drivers to something more recent. Official Asus support page only provides an ancient version from 2010. If you try to use the official and latest AMD drivers you will end up with a black screen during the installation process.

After a lot of digging on Google I found out that this is a widespread problem and the last driver from AMD that still work turns out to be 13.4, which is not ideal, but still better than what Asus provides. Sometimes after that apparently the back lighting of the screen is broken. Source.

Finding 13.4 drivers is a challenge. Guru3D used to have all versions listed on their websites, but AMD blocked external linking some time ago, so all links are a dead end. You also can't find 13.4 in AMD driver archives. So after scouring the internet, I finally found a working version. I am attaching it to this post for your leisure.

Download

Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs