Most of the time you don't want to write your own installer for distributing software. You use an off-the-shelf installation program that packages it all up for you instead.

But if you feel adventurous or need your installer to perform some very specific tasks you probably want to write your own. This post will shows you a few gotchas you might encounter along the way with Qt and WINAPI.

Qt Resource System

Qt resource system and it's .qrc files are perfect candidates for an installer. They allow you to package all the needed assets of your program into a single executable. If a .qrc file becomes too big however you might run into build errors like:

FatalError C1060:"compiler is out of heap space"

After hours of Googling for a solution it came down to this and this. Let me just say that increasing your Windows swap file and all the other solutions probably won't work. The easiest thing to do is to create another .qrc file and move some of the existing data files into it. If that one becomes too big you can simply create more of them.

This error can also appear if your project has too many header file inclusions. The .qrc files are being compiled into giant binary structs so it's the same problem in its core.

File permissions

This is an example code for copying an internal file to the filesystem:

QFile c(":\\somefile"); //internal file in .qrc
c.copy("path_in_filesystem");

But first we need to check whether a file already exists and remove it (think reinstall or install repair). This is needed because the copy() function has no "overwrite" parameter (which is stupid but that's Qt's fault):

QFile p("somepath/somefile");
if (p.exists()) {
  if (!p.remove()) {
    //could not remove file, permission denied
  }
}

This will fail because there is a small gotcha with the Qt resource system: all internal files are read-only! If you just copy an internal file to the filesystem it will keep it's read-only attribute and doing anything but reading the file will fail with Permission denied. Without knowing this simple and obvious fact you will lose hours trying to figure out what the hell is going on.

Setting write permissions after copy solves this problem, for example:

p.setPermissions(QFile::ReadOther|QFile::WriteOther);

Another gotcha with permissions is Windows UAC. Writing to Program Files needs admin permissions so your installer must be executed as administrator or request these privileges by default. Running it as a regular user will end up in a bunch of Permission denied errors.

The question then is, how to make your installer request admin privileges when executed. One way is to play with custom manifests and trying to integrate that with qmake… and probably fail on the end. Thankfully there is a simple solution by adding the following lines to qmake:

win32 {
    CONFIG += embed_manifest_exe
    QMAKE_LFLAGS_WINDOWS += /MANIFESTUAC:level=\'requireAdministrator\'
}

Make sure you enable UAC on your workstation computer so you can experience the same thing as your users. If you have it disabled you won't actually notice anything being wrong.

Registry

A proper installer should correctly register the program in the Add/Remove Programs and provide an uninstaller.

It basically has to write some registry entries. A great reference on required and optional entries can be found here. But there is a catch.. there is always a catch. The first one is that you probably can't use the QSettings class because it does not support additional CRegKey.Open() flags such as KEY_WOW64_64KEY. Your 32bit program should probably write to 64bit registry because that's what you see when you run regedit from the system. If you don't know the difference you might lose hours upon hours of trying to figure out why your registry is not changing while your WINAPI calls are working perfectly.

The gotcha is that the registry paths you need to write are different if your application is 32bit or 64bit.

For 32bit it is

Software\\Wow6432Node\\Microsoft\\Windows\\CurrentVersion\\Uninstall\\YourProduct

And for 64bit it is

Software\\Microsoft\\Windows\\CurrentVersion\\Uninstall\\YourProduct

To determine whether we are running under WOW64 or not there is the following code lurking on the internets:

BOOL IsWow64() {
  typedef BOOL (WINAPI *LPFN_ISWOW64PROCESS) (HANDLE, PBOOL);
  LPFN_ISWOW64PROCESS fnIsWow64Process;
  BOOL bIsWow64 = FALSE;
  //IsWow64Process is not available on all supported versions of Windows.
  //Use GetModuleHandle to get a handle to the DLL that contains the      function
  //and GetProcAddress to get a pointer to the function if available.
  fnIsWow64Process = (LPFN_ISWOW64PROCESS) GetProcAddress(
  GetModuleHandle(TEXT("kernel32")),"IsWow64Process");
  if(NULL != fnIsWow64Process) {
    if (!fnIsWow64Process(GetCurrentProcess(),&bIsWow64)) {
      //handle error
    }
  }
  return bIsWow64;
}

I have not yet tested whether CRegKey would open the correct version by default and also write to the correct place without setting any additional flags. Specifically, I am not sure whether Wow6432Node would be appended automatically.

Creating a shortcut

 Without losing too much words on this, creating a desktop shortcut is as simple as using QFile::link().

Rolling back

 User should be able to cancel the installation at any time and have its system intact. This means that we can't simply leave a bunch of files and registry entries on the system, we need to put everything back to the original state. I solved this problem by dividing each installation step in a separate method and then writing a rollback counterpart that does exactly the opposite. For every copied file I have a file removal and the same goes for every other modification. I also call the rollbacks if any of the installation steps fail.

If you are modifying existing files it is a good idea to first make a copy of the file and then place it back in case of a rollback.

Try to keep the system as intact as possible in case of a failure or cancellation.

Figuring out the system paths

 Let's say you want to copy your program files to C:/Program Files/Yourproject and you also want to create a desktop shortcut. You have two problems. First problem is that you don't know whether your folder should go to Program Files or Program Files (x86) because you don't know on what system you are. If your program is 32bit you know it should go to the (x86) but you don't know if that even exists – you might be on 32bit OS.

You also don't know the path to user's desktop. You have to use WINAPI and even that is a problem because Windows XP lacks some flags that newer systems have. I came up with the following function:

QString getProgramFiles() {
  TCHAR szPath[MAX_PATH];
  if (IsWinXP32()) {
      if(SUCCEEDED(SHGetFolderPath(NULL, CSIDL_PROGRAM_FILES, NULL, SHGFP_TYPE_DEFAULT, szPath))) {
        return QString::fromWCharArray(szPath);
      }
      else {
        return "";
      }
  }
  else {
    if(SUCCEEDED(SHGetFolderPath(NULL, CSIDL_PROGRAM_FILESX86, NULL, SHGFP_TYPE_DEFAULT, szPath))) {
       return QString::fromWCharArray(szPath);
    }
    else {
      return "";
    }
  }
}

The culprit is CSIDL_PROGRAM_FILESX86 which returns an empty string on Windows XP.

For finding the desktop you can use CSIDL_DESKTOP flag.

Determining 32bit Windows XP can be done like this:

bool IsWinXP32() {
  DWORD version = GetVersion();
  DWORD major = (DWORD) (LOBYTE(LOWORD(version)));
  DWORD minor = (DWORD) (HIBYTE(LOWORD(version)));
  return ((major == 5) && (minor == 1));
}

The uninstaller

Making the uninstaller is quite an easy task except for the last part – removing uninstaller itself. A running process cannot delete itself so this is quite an annoying problem. A few examples can be seen here. What I ended up doing is basically calling cmd.exe and doing a rmdir (which contains the uninstaller:

QStringList a;
a << "/c"; //command
a << "rd"; //rmdir
a << "/s/q"; //with subdirectories and silently
a << projectpath;

QProcess::startDetached(getSystem32()+"\\cmd.exe", a);
QApplication::quit();

where getSystem32 returns the path to System32 using the above mentioned CSIDL flags. This pops up a cmd window for a brief period after the uninstaller quits by itself.

These are a few problems I encountered along the way when working on my own installer. it is really not that difficult but there are a few subtle problems which might get you along the way, some inside Qt and others in WINAPI. Hopefully this can save someone a few hours combating the same issues if you happen to land on this blog. :)