All posts by cen

pgadmin4 on Fedora 25

wget https://download.postgresql.org/pub/repos/yum/9.6/fedora/fedora-25-x86_64/pgdg-fedora96-9.6-3.noarch.rpm
sudo dnf install pgdg-fedora96-9.6-3.noarch.rpm
sudo dnf install pgadmin4-v1
sudo systemctl start pgadmin4-v1

You can now access the web interface at http://localhost:5050.

Unfortunately the standalone app does not currently work due to a bug in pgadmin4 package.

Fortunately the pgadmin4 standalone app is just a web wrapper so you are not missing much.

Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

Keycloak OAuth endpoints for Postman/HTTP Clients

When testing REST services secured by Keycloak you need to retrieve access tokens via Postman or similar REST client. If you want to implement your own client that has to authenticate with a token you also need to know the Keycloak OpenID endpoints in order to retrieve the access token, refresh it or to end the session (logout).

Retreiving the tokens for a public client using username and password

Public client is typically used for web applications and other client side apps.

Method: POST
URL: https://keycloak.example.com/auth/realms/myrealm/protocol/openid-connect/token
Body type: x-www-form-urlencoded
Form fields:
client_id <my-client-name>
grant_type password
username <username>
password <password>

Retreiving the tokens for a confidential client using client secret

Confidential client is typically used for secure apps on the back-end.

Method: POST
URL: https://keycloak.example.com/auth/realms/myrealm/protocol/openid-connect/token
Body type: x-www-form-urlencoded
Form fields:
client_id <my-confidential-client-name>
grant_type client_credentials
client_secret <my-client-secret>

Retreive an access token with a refresh token

The first two methods will yield you an access token which you use in the Authorization HTTP header and a refresh token which you save for later. Refresh tokens have much longer expire time as access tokens. The idea is that when the access token expires you use the refresh token to get a new access token. This request also gives you a new refresh token so you can keep the session alive until maximum refresh token expire time is reached. Refresh token expire time equals the session expire time.

Method: POST
URL: https://keycloak.example.com/auth/realms/myrealm/protocol/openid-connect/token
Body type: x-www-form-urlencoded
Form fields:
client_id <my-client-name>
grant_type refresh_token
refresh_token <my-refresh-token>

Logout the session

To logout and invalidate the session, call a /logout endpoint with your refresh token. The validity of the refresh token is essentially the validity of your entire session.

Method: POST
URL: https://keycloak.example.com/auth/realms/myrealm/protocol/openid-connect/logout
Body type: x-www-form-urlencoded
Form fields:
client_id <my-client-name>
refresh_token <my-refresh-token>
Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

Fedora 25 on Lenoyo Y50

Everything except WiFi worked out of the box. To get the WiFi working:

sudo dnf install http://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-25.noarch.rpm
sudo dnf install http://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-25.noarch.rpm

# The rest
sudo dnf install kernel-devel broadcom-wl akmod-wl akmods
sudo akmods
sudo reboot

Taken from here.

In your BIOS make sure you disable Secure Boot. Not UEFI, not Legacy mode, specifically the switch that disables secure boot and nothing else. After these steps, WiFi works. To enter BIOS on Y50, tap F2 after Lenovo splash screen.

 

Edit 7.2.2017: WiFi performance is unfortunately ABYSMAL. Will update this post if I find any solutions. Connection is super slow and constantly dropping.

Luckily, USB tethering from Android works like a charm so it's not a deal breaker for now.

Edit 2: looks like blacklisting bcma driver makes things much much better:

sudo nano /etc/modprobe.d/blacklist.conf

#add this
blacklist bcma

sudo systemctl restart NetworkManager

 

Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

Expose your dev machine to the public via reverse SSH tunnel

Scenario: you are creating a REST service which needs to be exposed to the public even in early stage of development due to an upstream provider which sends back feedback data from a webhook API.

You are also behind a NAT so you'd have to port forward yourself out but you can't do that for whatever reason. Or maybe you are behind a firewall and 7 proxies.

All you need is an external server with a public IP. Then, on your machine:

ssh -R 0.0.0.0:8080:localhost:8080 server_user@public_server_ip

In the above example, I used port 8080 which my REST service uses when I develop. On your public server, make sure you have

GatewayPorts yes

in /etc/ssh/sshd_config. If it is missing, add it.

And that's it.. your local REST service is now publicly accessible via public_server_ip:8080.

a-m-a-z-i-n-g

Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

OJDBC7 in a Docker container? Prepare for trouble

Scenario: A JDK8 Docker container using OJDBC7 to connect to the database. Sounds simple enough, what could go wrong?

Simptoms: Connecting to the database randomly takes several minutes, fails with a weird SqlRecoverableException: no more data to read from socket or just works fine as if there is no problem.

The same Docker image also works fine on some machine but fails consistently on other.

The reason is this. Docker is not good at /dev/random. Probably even more so if you run it in a VM, since it's double isolated from actual entropy sources (my non scientific observation). For whatever reason, OJDBC defaults to /dev/random and this causes a block when connecting to the database due to high probability of /dev/random depletion.

Simple solution is to just mount /dev/urandom to /dev/random inside the Docker, in docker run command:

-v /dev/urandom:/dev/random

So.. if you ever want to use OJDBC inside Docker, remember this flag. It will save lives or at least spare you hours of useless debugging.

 

Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

JPQL: getting whole entities on a distinct column

If you want distinct entities in JPQL you would normally write something like this:

SELECT DISTINCT(l) FROM Letter l WHERE l.name=:name;

But this will do a DISTINCT on the whole Entity. This will also fail on Oracle DB if your Entity contains a CLOB. What if you really want to do a DISTINCT on a field, for example:

SELECT DISTINCT(l.id) FROM Letter l WHERE l.name=:name;

Unfortunately, this only returns an array of ID fields. If you want to retrieve the full entities and do a DISTINCT on a field the final query looks like:

SELECT ll FROM Letter ll WHERE ll.id IN (SELECT DISTINCT(l.id) FROM Letter l WHERE l.name=:name);
Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

Fedora 24 XFCE pains and gains

I recently installed Fedora 24 XFCE on my brand new Entroware Apollo, a Linux friendly laptop. Here is a list of problems I encountered during my first week of use. While most of these problems had a solution with a bit of googling, non-tech savvy person would have severe problems solving them.

PAINS

  1. System completely hangs when I connect a second monitor. Unresolved.
  2. Could not see any WiFi networks because the interface was not managed. Had to make it managed in NetworkManager.conf manually.
  3. At some point, XFCE panel would not show up anymore and I got an error message on startup asking me to start the panel. Solved it by deleting .config folder. What the hell?
  4. No login prompt when laptop comes out of suspend, even though I have both option in power management and session settings turned on. Unresolved.
  5. Adding programs to favorites in Whiskers menu sometimes does not persist across reboots. I think esepcially when I did a hard reset due to issue #1.
  6. My Nexus 4 would not automount in Thunar over USB. After an hour of Googling and installing random packages I got it to work, I think?

GAINS

  1. Numix theme on XFCE is extremely nice, I love it. There is just one small bug so far in Volume widget, the selection color hides the slider completely.
  2. My FreeNAS was automatically detected by Thunar while the old Gnome-files setup would not show it and even refuse to mount it sometimes manually. A nice surprise.
  3. I like how you can fully customize XFCE panels. The only thing I actually couldn't do is completely hide the panel (there is like a 3px grey bar when the panel is hidden). Other than that it's super nice.
  4. Fedy is super nice. Props to it's maintainers, it all just works.

 

So basically, everything kinda works now but it was a bumpy road to achieve this. Hopefully I get some feedback on issue #1 in Fedora bug tracker because it's really severe.

Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

StormLib and BNCSUtil now available from repositories

Update 2023: repos are currently unavailable

 

Finally after years and years of screwing around with building StormLib or BNCSUtil everytime you moved your ghost bots to another server is finally over. As of today, I am hosting both libraries in apt and rpm repositories. In the process I also discovered that StormLib managed to get into Debian Testing (https://packages.debian.org/source/stretch/stormlib) so let's hope it'll be included in the next stable release!

I also  pushed CPack generator code into both repositories, so you can build these packages yourself. BNCSUtil was forked and cleaned especially.

DEB based distros

  1. To /etc/apt/sources.list add:
    #apt.xpam.pl
    deb https://apt.xpam.pl/ jessie main
  2. Add GPG key:
    wget -qO - https://apt.xpam.pl/xpam.pl-pubkey.asc | sudo apt-key add -
  3. Update and install:
    sudo apt-get update
    sudo apt-get install stormlib bncsutil

RPM based distros

  1. In /etc/yum.repos.d/rpm.xpam.pl.repo add
    [rpm.xpam.pl]
    name = rpm.xpam.pl
    baseurl = https://rpm.xpam.pl
    

    or, with dnf on Fedora:

    dnf config-manager --add-repo https://rpm.xpam.pl
  2. If using dnf, make sure to import the public key:
    rpm --import https://rpm.xpam.pl/rpm-pubkey.asc
  3. Update and install:
    yum update && yum install stormlib bncsutil

    or dnf equivalent.

 

This was a nice project to spend a few weekends on but there are more plans in 2016 to clean and tidy up the bnet tools we all use and love:
-add CMake and CPack support to our own bots, link against upstream StormLib and release a cleaned up ghost code, ease the setup process via repo installs with full dependency resolution (mysql, boost, storm and bncsutil). Provide precompiled binaries for Windows.
-provide patches to some other popular ghost forks to work with upstream StormLib.
-package PvPGN?
-provide pkgng packages for FreeBSD. This will probably involve writing pkgng generator for CMake's CPack which could be an interesting side project in itself.

Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

Qt 5 + MSVC 2015 + OpenSSL + Windows XP Support = absolute mess

Recently I decided to move all of our C++/Qt based projects up to the latest version of Qt and C++ toolkits. That means Qt 5, Visual Studio 2015 and all the latest libraries which are needed either for Qt or standalone. There was one caveat tho.. we still need to support Windows XP because a large amount of our player base still uses it.  The nightmare begins.

This guide talks about:

-building shared release libraries of OpenSSL with msvc2015 XP target

-build shared release Qt framework with msvc2015 XP target

-build your application with msvc2015 XP target

1. Getting OpenSSL (1.0.1.p)

Building OpenSSL with MSVC 2015 and Windows XP target is an absolute clusterfuck. Forget XP target, even without that it simply WON'T compile with the usual instructions which used to work in the past (ex: http://developer.covenanteyes.com/building-openssl-for-visual-studio/). Thankfully, I found pre-built binaries and a useful batch script at http://www.npcglib.org/~stathis/blog/precompiled-openssl/. The pre-built binaries are ok but they are not built with XP target. They are also built with MT/MD naming which Qt does not really like (Qt insists that you link to libeay32 and ssleay32 and nothing else). Simply renaming them does not really work because .lib tries to include the wrong thing and you end up needing to provide both (no go).

So after a full day of working around the various problems I managed to compile OpenSSL with msvc2015 and XP target. This is a rough procedure so I might have missed something because I recompiled at least 100 times before it worked, but it should at least guide you on the right track:

  • Modify the batch script from the link above (you need Cygwin, perl.. ) and make it to a working state
  • Run these in cmd: http://doc.qt.io/qt-5/windows-issues.html (make sure you have Windows SDK installed and run the 2015 vcvarsall.bat version of the script, obviously)
  • Open ms/ntdll-x86.mk that is generated
  • Add the same env vars you just added in cmd to the top of makefile:
  • PATH=C:\Program Files (x86)\Microsoft SDKs\Windows\v7.1A\Bin;$(PATH)
    INCLUDE=C:\Program Files (x86)\Microsoft SDKs\Windows\v7.1A\Include;$(INCLUDE)
    LIB=C:\Program Files (x86)\Microsoft SDKs\Windows\v7.1A\Include;$(LIB)
  • Add
    -D_USING_V110_SDK71_

    to CFLAGS

  • Add
    -I"C:\Program Files (x86)\Microsoft SDKs\Windows\v7.1A\Include" -I"C:\Program Files (x86)\Microsoft SDKs\Windows\v7.1A\Include"

    to CFLAGS

  • Replace /subsystem flag in LFLAGS with /subsystem:console,5.01
  • Also add the same subsystem flag to MKLIB and MKFLAGS
  • Remove MD from SSL and CRYPTO flags so you get libeay32 and libssl32 named binaries
  • In root OpenSSL folder run:
     nmake -f ms/ntdll-x86.mak

    This should produce XP targeted OpenSSL libraries. You know they are XP targeted if you open both DLLs in HEX editor and check rows 130-150, somewhere in those rows you need to se 05 00 01 … appear twice. See: http://www.tripleboot.org/?p=423

2. Build Qt with Windows XP support

Setup the environment again: http://doc.qt.io/qt-5/windows-issues.html

Then configure Qt, I personally used:

configure -target xp -release -qmake -opensource -nomake examples -opengl desktop -platform win32-msvc2015 -openssl -I C:\Users\me\git\openssl-1.0.1p-vs2015\include -L C:\Users\me\git\openssl-1.0.1p-vs2015\shared OPENSSL_LIBS="-llibeay32 -lssleay32 -lgdi32"

where -I and -L flags need to point to OpenSSL include files and the libraries you just built in previous step. We are doing a shared build (I used to like it all static but it turns out shipping Qt libraries bundled in the .exe with every single application update is a complete waste of bandwidth and time, at least 10MB every time).

Then build the Qt with: jom
Do not use nmake because jom is much faster!

After Qt builds, add it as a Kit in Qt Creator. You need to specify path to qmake.exe which is in qt-everywhere-opensource-src-5.X.X\qtbase\bin.

4. Configuring your Application Qmake

Now to your actual project. Add this to your qmake (.pro) file:

DEFINES += _ATL_XP_TARGETING
DEFINES += PSAPI_VERSION=1
QMAKE_LFLAGS_WINDOWS = /SUBSYSTEM:WINDOWS,5.01

Second one? Because reasons: http://blogs.msdn.com/b/vcblog/archive/2009/08/27/windows-sdk-v7-0-v7-0a-incompatibility-workaround.aspx

5. Deployment

Add all Qt DLL libraries in the same folder as your .exe (from qtbase/lib), depending on the modules, it is usually at least:

QtCore.dll, QtGui.dll, QtNetwork.dll, QtWidgets.dll…

Also do not forget to include "platforms/qwindows.dll" in the same folder as your .exe. You also need to provide ssleay32.dll and libeay32.dll the same way. You might also need to provide msvc C++ redistributable dll files but I usually just instruct users to install the full C++ redistributable package from Microsoft.

 

And there you have it, Qt 5 with OpenSSL built with MSVC 2015 and with Windows XP support. An absolute mess if you are starting from zero and don't know all the little annoyances but once you do it's a relief.

Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs