Fedora 24 XFCE pains and gains

I recently installed Fedora 24 XFCE on my brand new Entroware Apollo, a Linux friendly laptop. Here is a list of problems I encountered during my first week of use. While most of these problems had a solution with a bit of googling, non-tech savvy person would have severe problems solving them.

PAINS

  1. System completely hangs when I connect a second monitor. Unresolved.
  2. Could not see any WiFi networks because the interface was not managed. Had to make it managed in NetworkManager.conf manually.
  3. At some point, XFCE panel would not show up anymore and I got an error message on startup asking me to start the panel. Solved it by deleting .config folder. What the hell?
  4. No login prompt when laptop comes out of suspend, even though I have both option in power management and session settings turned on. Unresolved.
  5. Adding programs to favorites in Whiskers menu sometimes does not persist across reboots. I think esepcially when I did a hard reset due to issue #1.
  6. My Nexus 4 would not automount in Thunar over USB. After an hour of Googling and installing random packages I got it to work, I think?

GAINS

  1. Numix theme on XFCE is extremely nice, I love it. There is just one small bug so far in Volume widget, the selection color hides the slider completely.
  2. My FreeNAS was automatically detected by Thunar while the old Gnome-files setup would not show it and even refuse to mount it sometimes manually. A nice surprise.
  3. I like how you can fully customize XFCE panels. The only thing I actually couldn't do is completely hide the panel (there is like a 3px grey bar when the panel is hidden). Other than that it's super nice.
  4. Fedy is super nice. Props to it's maintainers, it all just works.

 

So basically, everything kinda works now but it was a bumpy road to achieve this. Hopefully I get some feedback on issue #1 in Fedora bug tracker because it's really severe.

Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

StormLib and BNCSUtil now available from repositories

Finally after years and years of screwing around with building StormLib or BNCSUtil everytime you moved your ghost bots to another server is finally over. As of today, I am hosting both libraries in apt and rpm repositories. In the process I also discovered that StormLib managed to get into Debian Testing (https://packages.debian.org/source/stretch/stormlib) so let's hope it'll be included in the next stable release!

I also  pushed CPack generator code into both repositories, so you can build these packages yourself. BNCSUtil was forked and cleaned especially.

DEB based distros

  1. To /etc/apt/sources.list add:
  2. Add GPG key:
  3. Update and install:

RPM based distros

  1. In /etc/yum.repos.d/rpm.xpam.pl.repo add

    or, with dnf on Fedora:
  2. If using dnf, make sure to import the public key:
  3. Update and install:

    or dnf equivalent.

 

This was a nice project to spend a few weekends on but there are more plans in 2016 to clean and tidy up the bnet tools we all use and love:
-add CMake and CPack support to our own bots, link against upstream StormLib and release a cleaned up ghost code, ease the setup process via repo installs with full dependency resolution (mysql, boost, storm and bncsutil). Provide precompiled binaries for Windows.
-provide patches to some other popular ghost forks to work with upstream StormLib.
-package PvPGN?
-provide pkgng packages for FreeBSD. This will probably involve writing pkgng generator for CMake's CPack which could be an interesting side project in itself.

Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

Qt 5 + MSVC 2015 + OpenSSL + Windows XP Support = absolute mess

Recently I decided to move all of our C++/Qt based projects up to the latest version of Qt and C++ toolkits. That means Qt 5, Visual Studio 2015 and all the latest libraries which are needed either for Qt or standalone. There was one caveat tho.. we still need to support Windows XP because a large amount of our player base still uses it.  The nightmare begins.

This guide talks about:

-building shared release libraries of OpenSSL with msvc2015 XP target

-build shared release Qt framework with msvc2015 XP target

-build your application with msvc2015 XP target

1. Getting OpenSSL (1.0.1.p)

Building OpenSSL with MSVC 2015 and Windows XP target is an absolute clusterfuck. Forget XP target, even without that it simply WON'T compile with the usual instructions which used to work in the past (ex: http://developer.covenanteyes.com/building-openssl-for-visual-studio/). Thankfully, I found pre-built binaries and a useful batch script at http://www.npcglib.org/~stathis/blog/precompiled-openssl/. The pre-built binaries are ok but they are not built with XP target. They are also built with MT/MD naming which Qt does not really like (Qt insists that you link to libeay32 and ssleay32 and nothing else). Simply renaming them does not really work because .lib tries to include the wrong thing and you end up needing to provide both (no go).

So after a full day of working around the various problems I managed to compile OpenSSL with msvc2015 and XP target. This is a rough procedure so I might have missed something because I recompiled at least 100 times before it worked, but it should at least guide you on the right track:

  • Modify the batch script from the link above (you need Cygwin, perl.. ) and make it to a working state
  • Run these in cmd: http://doc.qt.io/qt-5/windows-issues.html (make sure you have Windows SDK installed and run the 2015 vcvarsall.bat version of the script, obviously)
  • Open ms/ntdll-x86.mk that is generated
  • Add the same env vars you just added in cmd to the top of makefile:
  • Add

    to CFLAGS
  • Add

    to CFLAGS
  • Replace /subsystem flag in LFLAGS with /subsystem:console,5.01
  • Also add the same subsystem flag to MKLIB and MKFLAGS
  • Remove MD from SSL and CRYPTO flags so you get libeay32 and libssl32 named binaries
  • In root OpenSSL folder run:

    This should produce XP targeted OpenSSL libraries. You know they are XP targeted if you open both DLLs in HEX editor and check rows 130-150, somewhere in those rows you need to se 05 00 01 … appear twice. See: http://www.tripleboot.org/?p=423

2. Build Qt with Windows XP support

Setup the environment again: http://doc.qt.io/qt-5/windows-issues.html

Then configure Qt, I personally used:

where -I and -L flags need to point to OpenSSL include files and the libraries you just built in previous step. We are doing a shared build (I used to like it all static but it turns out shipping Qt libraries bundled in the .exe with every single application update is a complete waste of bandwidth and time, at least 10MB every time).

Then build the Qt with: jom
Do not use nmake because jom is much faster!

After Qt builds, add it as a Kit in Qt Creator. You need to specify path to qmake.exe which is in qt-everywhere-opensource-src-5.X.X\qtbase\bin.

4. Configuring your Application Qmake

Now to your actual project. Add this to your qmake (.pro) file:

Second one? Because reasons: http://blogs.msdn.com/b/vcblog/archive/2009/08/27/windows-sdk-v7-0-v7-0a-incompatibility-workaround.aspx

5. Deployment

Add all Qt DLL libraries in the same folder as your .exe (from qtbase/lib), depending on the modules, it is usually at least:

QtCore.dll, QtGui.dll, QtNetwork.dll, QtWidgets.dll…

Also do not forget to include "platforms/qwindows.dll" in the same folder as your .exe. You also need to provide ssleay32.dll and libeay32.dll the same way. You might also need to provide msvc C++ redistributable dll files but I usually just instruct users to install the full C++ redistributable package from Microsoft.

 

And there you have it, Qt 5 with OpenSSL built with MSVC 2015 and with Windows XP support. An absolute mess if you are starting from zero and don't know all the little annoyances but once you do it's a relief.

Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

Rode NT-USB Linux support

Googling around, I could not find a single mention whether Rode NT-USB microphone supports Linux or not. After some semi-encouraging words from Rode support I decided to go ahead and buy it anyway. I am happy to announce that it works out of the box and so far I did not encounter any problems on Fedora 23. The microphone is properly detected in Pavucontrol and Open Broadcaster. I am a complete amateur but to me the sound is crisp clear with zero background noise. The comparison with my old Siberia V2 headphone microphone or my in-built laptop one is night and day. These two would produce a lot of noise as soon as increasing the volume over 100% (at 100% they would be way too silent) and it was a complete disaster. NT-USB can go much louder and has no background noise even sitting 40cm from my loud laptop fan.

Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

The state of SVG rendering

This is an interesting little experiment I did when I was searching for a FreeBSD SVG logo to put on a poster. As you might know, SVG format is for vector images, allowing for infinite scaling without pixelation, which is exactly what you need, if you are going to print something on a big quality poster. Either that or you need originals in huge sizes for downscaling.

What caught my eye when I opened the image with Firefox was the lack of detail inside the devil ball which should be visible. This sent me on a little crusade to find out which browsers and image programs are actually capable of rendering it correctly.

For (assumingly) correct reference render we will take the logo from the freebsd.org:

Official FreeBSD logo as seen on freebsd.org

 

Now let's jump right into it..

FreeBSD logo rendered by Firefox
FreeBSD logo rendered by Firefox

Firefox render is very basic, lacks most of the highlights and internal details.

FreeBSD logo rendered by Chrome
FreeBSD logo rendered by Chrome

Chrome takes some artistic freedoms and completely screws up in the process. The elements are there but layered improperly plus the horns are now black. Obvious transparency and overlay issues.

FreeBSD logo rendered by IE
FreeBSD logo rendered by IE

Internet Explorer manages to produce the worst render of all programs.

FreeBSD logo rendered by Opera
FreeBSD logo rendered by Opera

Opera follows Chrome since they use the same underlying engine.

FreeBSD logo rendered by Safari
FreeBSD logo rendered by Safari

Safari also falls into the Webkit family with Chrome and Opera.

FreeBSD logo rendered by svg-edit
FreeBSD logo rendered by svg-edit

Svg-edit is an online JavaScript based tool.

FreeBSD logo rendered by Imagemagick
FreeBSD logo rendered by ImageMagick

ImageMagick is very close but it has an extra bright ring going through the middle which apparently shouldn't be there.

FreeBSD logo rendered by Inkscape
FreeBSD logo rendered by Inkscape

Inkscape is the only tool in this test that produced a proper render.

FreeBSD logo rendered by Gimp
FreeBSD logo rendered by Gimp

Gimp is also very close but has the same imperfection as ImageMagick.

FreeBSD logo rendered by IrfanView
FreeBSD logo rendered by IrfanView

IrfanView uses an external plugin to render SVGs that is not free. If we ignore the overlay text for a moment, the image suffers from Chrome-like problems except it's of absolute terrible quality.

 

And there you have it, the piss poor state of SVG rendering as of July 2015.

SVG protip

If you want to resize an SVG image and produce a high resolution PNG, the easiest method is to use ImageMagick from command line:

Density is specified in dpi by default. You determine the dpi based on the size of the print you actually want to produce.

 

Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

Black screen with Catalyst > 13.4 on Asus N71jq and similar

If you own an Asus N71jq or a similar laptop model from 5 years ago you probably tried to update your Catalys drivers to something more recent. Official Asus support page only provides an ancient version from 2010. If you try to use the official and latest AMD drivers you will end up with a black screen during the installation process.

After a lot of digging on Google I found out that this is a widespread problem and the last driver from AMD that still work turns out to be 13.4, which is not ideal, but still better than what Asus provides. Sometimes after that apparently the back lighting of the screen is broken. Source.

Finding 13.4 drivers is a challenge. Guru3D used to have all versions listed on their websites, but AMD blocked external linking some time ago, so all links are a dead end. You also can't find 13.4 in AMD driver archives. So after scouring the internet, I finally found a working version. I am attaching it to this post for your leisure.

Download

Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

Periodically pause and resume Transmission torrents on FreeNAS

Unless you have great internet bandwidth you probably don't want to leech and seed your 100 torrents during the day. You usually want them to seed during the night and then pause in the morning. Let's do this on our FreeNAS box.

  1.  Open up your jails, select transmission and fire up the shell from the icon below.
  2. If you dislike vi as default editor, set it to ee by running the following command:

    To make this choice persistent, open up .cshrc and add the command on the bottom.
  3. Run

    and write the following lines:

    Change the username and password with the ones you use to access Transmission WebUI. The example will start all torrents at 1AM and stop them at 7AM.

 

Update in 2023: in TrueNAS 13 (perhaps even earlier versions), transmission-remote is no longer preinstalled inside the jail, you need to install it first:

 

Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

My personal FreeNAS build and review

This post will talk about how I built my own FreeNAS box, including a short review of the components used. I will also try to mention all the problems and pitfalls that I can remember.

Components used

Component mini reviews

ASrock C2550D4I

This motherboard is the perfect choice for a NAS box. It has a low power Atom processor, passively cooled, micro ITX and most importantly.. 12 freaking SATA ports. Which means you won't even need an add-on card for the build. It also comes with 2 Ethernet ports and a dedicate IPMI.


The good:

  • low power, silent
  • 12 SATA ports
  • 2 Gigabit LAN ports and dedicated IPMI port
  • nice web management interface
  • good manual
  • no other manufacturer provides anything similar to this motherboard

The bad:

  • no USB 3.0 header which means the front USB on DS380 is completely useless (unless you get a PCI-E USB controller)
  • only 3 USB 2.0 ports, either 1 front and 2 back or vice-versa (again, unless you get a PCI-E USB controller)
  • in configuration with DS380, the ATX power connector is on the "wrong" side which means that the ATX cable has to go all across the motherboard. In a normal computer case this wouldn't be an issue though
  • price

Bough on amazon.de for 301€

 

SilverStone DS380

This case is pretty much perfect for a NAS box. It has eight 3.5" hot swap bays and additional 4 internal 2.5" bays. You also don't need to worry about providing 12 SATA power cables because it has a circuit with 2 molex input connectors which power all 8 hot-swap bays.. no Y splitters needed.

 

 

The good:

  • eight 3.5" hot-swap bays and four 2.5" internal bays
  • premium look-and-feel
  • removable dust filters on all intakes
  • hot-swap circuit which also supplies power
  • the 2.5" cage is removable
  • front USB and sound is removable

The bad:

  • the fans are not the most silent on the planet
  • the manual is not the most accurate
  • the hot-swap circuit has two fan headers for the two side fans but if you use them, you can't control them from the BIOS at all
  • there is nothing to soften the vibrations from the drives in the caddies or the cage. Putting your hand on the case noticeably silences the system so improvements could be made.
  • price

Bought from local online shop for 182€

SilverStone SST-ST30SF

We need a SFX power supply for our case and there are not that many on the market to choose from. This is a very silent 300W power supply which is more than enough for everything we need. It just happened to be in my local online shop, the price was good and it matched the case so I went with it.

The good:

  • so far I haven't even seen the fan to spin up so this is effectively a passively cooled power supply
  • premium look-and-feel

The bad:

  • the ATX cable could be longer. It barely reaches the motherboard connector which means you can't really move it away if something else is blocking the path.. and since it goes all across the motherboard it bumps into the USB stick that holds FreeNAS, at least in my case.
  • it would be great if this was modular because we don't really need anything but ATX and 2x molex for the build (unless you use the internal bays)

Bought from local online shop for 60€

Crucial 8GB ECC DDR3-1600 CT102472BD160B

The only reason I went with this is because it was on ASrock memory QVL list and one of the only 8GB sticks from a somewhat reputable manufacturer. I got only one due to budget constraints but there are 3 slots left to upgrade at any time.

Bought from local online shop for 90€

 

4x HGST Deskstar NAS 4TB

I bough 4 of these, mainly because they are from HGST which I had a great experience with so far and the fact that they are 7200 RPM NAS drives. Online reviews also hinted that they beat WD Reds and Seagate NAS drives so I went with them.

The good:

  • HGST
  • price

The bad:

  • 4 of these spinning cause a lot of noise and you can hear them from quite far away. This is also the main source of noise in the build, everything else is relatively silent.

Bough from local online shop for 480€

 

Lian-Li PT-FN03

I bough this fan controller mainly because it fits in the PCI slot while most other controllers fit into the front of the case. The reason why I even got this is because I could not control the fans from BIOS at all so they were spinning at full speed all the time. But on the end it did not really matter much because the 4 disks are so loud that you don't even care about the fan noise. Looking from that perspective it was probably a failed purchase. Other than that it works ok and it comes with a molex splitter so you are not losing any connectors from your power supply.

Bough from amazon.de for 15€

 

USB header adapter and stick

If you want to put the USB stick with FreeNAS inside the case you need to buy a USB header to female (normally 2x female) adapter. By default, the ASrock motherboard has 2 USBs active at the back and one at the front header, so only one of the female connectors on the adapter will actually work. If you can't see the USB stick as bootable option in the BIOS, this might be the problem. Simply put the USB stick in the other slot.

Total cost

Roughly 1200€

The build

Fan controller was not used in this guide.

  1. Open the case, remove both cages. Optionally remove the front IO if you won't need it
  2. Mount the power supply
  3. Mount the motherboard
  4. Insert the RAM
  5. Plug in all the fans, front IO and ATX power
  6. Plug in all SATA cables into the motherboard. I suggest you start with the Intel ones (see manual for which is which). If you use regular HDDs use SATA 2 first and leave SATA 3 for the future (perhaps for SSDs).
  7. Plug in the USB header adapter and your FreeNAS stick. Note that if you have header->2x female adapter, one of them will be disabled by default.
  8. Put the 3.5" cage back in, leave the 2.5" one if you won't use it at this point
  9. Plug 2x molex and SATA cables into the cage
  10. Put the hard drives in caddies and insert them
  11. You should now be ready to boot. Enter BIOS, set the USB as first boot option and off you go

One thing I am not 100% sure is how to access the BIOS through IPMI to set the boot options (so we don't have to plug in the monitor and keyboard). The management software on the motherboard should have a default IP out of the box but I am not sure how you get to know the IP without checking it out in the BIOS. Tips welcome.

Other than that, the Megarac Management Platform that ASrock provides is very nice. It allows you to see the console output and power ON/OFF the server from your workstation.

Once FreeNAS boots it will output the web UI IP in the console. The other way you can figure out the IP is to open your router and check the DHCP table.

 

Conclusion

I ended up with creating a raidz2 zpool which turns out to be roughly 8TB of empty space. The reason I chose raidz2 is because when a disk dies I can comfortably send it to repair or buy a new one. This can take a few weeks so running in degraded state in case of raidz1 is probably not the smartest thing to do.

There are a few things that can be extended and improved in the future. I still have place for four 3.5" drives and since the single drive sizes are already at 6TB I will probably just create a new zpool when I need it. I also have four 2.5" bays still available inside the case so I could potentially buy SSDs when they become even cheaper and have greater capacities. 24GB of RAM can also be added.

The PCI-E 8x slot is unused at the moment. I could throw a GPU in there and have the box work as a HTPC machine although the noise from the hard drives would not be welcome anywhere near TV. The slot could also be used for additional USB 3.0 headers, so the front panel would actually be usable.

With all this expandability in mind, I am probably good for 5+ years in terms of storage.

 

 

Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs

Writing a Windows installer using Qt

Most of the time you don't want to write your own installer for distributing software. You use an off-the-shelf installation program that packages it all up for you instead.

But if you feel adventurous or need your installer to perform some very specific tasks you probably want to write your own. This post will shows you a few gotchas you might encounter along the way with Qt and WINAPI.

Qt Resource System

Qt resource system and it's .qrc files are perfect candidates for an installer. They allow you to package all the needed assets of your program into a single executable. If a .qrc file becomes too big however you might run into build errors like:

After hours of Googling for a solution it came down to this and this. Let me just say that increasing your Windows swap file and all the other solutions probably won't work. The easiest thing to do is to create another .qrc file and move some of the existing data files into it. If that one becomes too big you can simply create more of them.

This error can also appear if your project has too many header file inclusions. The .qrc files are being compiled into giant binary structs so it's the same problem in its core.

File permissions

This is an example code for copying an internal file to the filesystem:

But first we need to check whether a file already exists and remove it (think reinstall or install repair). This is needed because the copy() function has no "overwrite" parameter (which is stupid but that's Qt's fault):

This will fail because there is a small gotcha with the Qt resource system: all internal files are read-only! If you just copy an internal file to the filesystem it will keep it's read-only attribute and doing anything but reading the file will fail with Permission denied. Without knowing this simple and obvious fact you will lose hours trying to figure out what the hell is going on.

Setting write permissions after copy solves this problem, for example:

Another gotcha with permissions is Windows UAC. Writing to Program Files needs admin permissions so your installer must be executed as administrator or request these privileges by default. Running it as a regular user will end up in a bunch of Permission denied errors.

The question then is, how to make your installer request admin privileges when executed. One way is to play with custom manifests and trying to integrate that with qmake… and probably fail on the end. Thankfully there is a simple solution by adding the following lines to qmake:

Make sure you enable UAC on your workstation computer so you can experience the same thing as your users. If you have it disabled you won't actually notice anything being wrong.

Registry

A proper installer should correctly register the program in the Add/Remove Programs and provide an uninstaller.

It basically has to write some registry entries. A great reference on required and optional entries can be found here. But there is a catch.. there is always a catch. The first one is that you probably can't use the QSettings class because it does not support additional CRegKey.Open() flags such as KEY_WOW64_64KEY. Your 32bit program should probably write to 64bit registry because that's what you see when you run regedit from the system. If you don't know the difference you might lose hours upon hours of trying to figure out why your registry is not changing while your WINAPI calls are working perfectly.

The gotcha is that the registry paths you need to write are different if your application is 32bit or 64bit.

For 32bit it is

And for 64bit it is

To determine whether we are running under WOW64 or not there is the following code lurking on the internets:

I have not yet tested whether CRegKey would open the correct version by default and also write to the correct place without setting any additional flags. Specifically, I am not sure whether Wow6432Node would be appended automatically.

Creating a shortcut

 Without losing too much words on this, creating a desktop shortcut is as simple as using QFile::link().

Rolling back

 User should be able to cancel the installation at any time and have its system intact. This means that we can't simply leave a bunch of files and registry entries on the system, we need to put everything back to the original state. I solved this problem by dividing each installation step in a separate method and then writing a rollback counterpart that does exactly the opposite. For every copied file I have a file removal and the same goes for every other modification. I also call the rollbacks if any of the installation steps fail.

If you are modifying existing files it is a good idea to first make a copy of the file and then place it back in case of a rollback.

Try to keep the system as intact as possible in case of a failure or cancellation.

Figuring out the system paths

 Let's say you want to copy your program files to C:/Program Files/Yourproject and you also want to create a desktop shortcut. You have two problems. First problem is that you don't know whether your folder should go to Program Files or Program Files (x86) because you don't know on what system you are. If your program is 32bit you know it should go to the (x86) but you don't know if that even exists – you might be on 32bit OS.

You also don't know the path to user's desktop. You have to use WINAPI and even that is a problem because Windows XP lacks some flags that newer systems have. I came up with the following function:

The culprit is CSIDL_PROGRAM_FILESX86 which returns an empty string on Windows XP.

For finding the desktop you can use CSIDL_DESKTOP flag.

Determining 32bit Windows XP can be done like this:

 

The uninstaller

Making the uninstaller is quite an easy task except for the last part – removing uninstaller itself. A running process cannot delete itself so this is quite an annoying problem. A few examples can be seen here. What I ended up doing is basically calling cmd.exe and doing a rmdir (which contains the uninstaller:

where getSystem32 returns the path to System32 using the above mentioned CSIDL flags. This pops up a cmd window for a brief period after the uninstaller quits by itself.

 

These are a few problems I encountered along the way when working on my own installer. it is really not that difficult but there are a few subtle problems which might get you along the way, some inside Qt and others in WINAPI. Hopefully this can save someone a few hours combating the same issues if you happen to land on this blog. :)

 

Cen
GitHub
Eurobattle.net
Lagabuse.com
Bnetdocs