An Interview with Peter Mulholland, ex-VP

We tend to regard Feral and Aspyr as the key porting companies in the Linux world, yet this would be reductive. More individuals and companies dedicate themselves to porting games on Linux, infrequently making headlines. But we ought not discount companies such as Virtual Programming, aka VP, since their contributions from AAA games to classic titles continue to this day.

For a long time I wanted to figure out what went on at VP. They have proved elusive, avoiding direct communication following the conditions of Witcher 2‘s release. By all means I wanted to reach Peter Mulholland, key contributor to multiple VP ports, who left VP and moved on to other ventures in the recent past. Despite Peter initially unenthusiastic about the prospect, he ultimately accepted to answer a handful of questions. Please consider his answers as his personal opinions, in no way representative of his previous employer Virtual Programming’s.

Peter started his career with Amiga ports, long before Linux was even a platform.

Peter Mulholland: I worked at Runesoft before VP. I got that job after quitting University because I knew Paul Burkey, author of the Amiga game “Foundation“, for which Runesoft was the publisher. He had been asked to look at a Mac/Amiga port of “Simon the Sorcerer 2” but he did not have Intel assembly language experience. So we worked on it together (and re-ported 90% of it). We then did several ports for Runesoft.. The Feeble Files, Northland, Airline Tycoon Deluxe.. and some others I don’t remember. We had our own library which was somewhat like SDL at the time (but before SDL was developed) so it was not much effort to port some of those to Linux as well. I did those using Ubuntu 9.04 on an old Pentium 3 system I had spare. When VP got the Linux Witcher 2 port, my experience of Linux game porting (and my sysadmin skills) came in useful. I did the initial ground work of bringing across eON (which was Mac only) to Linux. Others then worked on the tech from there. I was the release engineer and the Steamworks admin for VP as well.

Peter described VP as “small and dedicated”. Despite the organization size, VP has ported several AAA titles to Linux over the years. One of their key advantages is a technology called eON to convert on the fly DirectX calls to OpenGL ones. This resembles WINE in theory, while it shares none of its code. eON remains a completely unrelated project, solely focused on the graphics part, unlike WINE which supports typical Windows desktop applications.

Peter: I cannot go into in depth details, but in ways it is similar to Wine, except it doesn’t aim for 100% Windows emulation. Wine implements a lot of the layers of Windows 1:1, for example that DLL’s such as kernel32 end up making calls to ntdll, which makes “syscalls” and so on. eON doesn’t do that, because it’s not really necessary for a game to run. So in that way, it is more like Feral’s IndirectX library (From what we have heard of it ). There is no code taken from WINE at all, it is all written from scratch. It started when one of our programmers thought “What happens if I load a PE (Portable Executable) file into an OSX process memory space, and jump into it?”

eON can work either at compile time, or without the original game source code. Apparently, making a port directly from the source code doesn’t necessarily change the performance profile, and one may choose to leave the original game code intact for multiple reasons.

Peter: Correct… it can be linked in at compile time if needed. However there’s usually very little reason to do this – it is better if the original game code stays built with its original toolchain. At the end of the day it’s Intel assembly code being run, so it really makes no speed difference if GCC compiled that code, or MSVC compiled that code. API calls are just a single JMP instruction, so it makes no difference if that instruction was put there by the compiler, linker, or by a custom PE loader.

 

A lot of Linux people make a big thing over us running code from a PE file, in reality it makes absolutely no difference performance wise. There are custom hooks into eON to improve performance that can be used from the Windows side of the game, even though the original game code is still being built with MSVC.

eON tech worked wonders for specific ports. For example. Dirt Showdown ran so smoothly with eON that it matched the Windows performance profile and exceeded it in specific cases.

Dirt Showdown constitutes an exception: everything depends heavily on the way in which original graphics API calls are written.

Peter: It really depends on the underlying game engine. There are some usage patterns of D3D which translate very well, and some which don’t. For example Saints Row was very drawcall heavy – no problem for D3D, but a sore point for GL. Vulkan might have made that better, but it was not available then. Heavily rewriting the SR engine (we had the source code) was out of the question on time and cost grounds.

Let’s turn to Witcher 2. As with other games from that timeframe, its release was triggered by Valve’s Steam machines initiative.

Peter: We had previously done the Mac port of Witcher 2 for them though, so I guess it made sense for us to do the Linux port. The developers (CDPR) were interested in the platform, and the business side were quite interested in Steam Machines as a console platform. I think it’s fair to say most of the Linux port work we did at VP (e.g. Bioshock Infinite, Saints Row series) were spurred on by the Steam Machines. Publishers lost interest after the Steam Machines didn’t work out. Valve really blew it IMO – however Linux wasn’t really ready either, with the poor AMD driver situation amongst other things. AMD APU’s would have been the ideal platform for a console.

As to why Valve was so interested to support Linux in the first place, Peter’s guess is as good as any: the results of a flat organization?

Peter: I think it is a core of Valve employees who are just Linux enthusiasts in general… due to their “flat” structure they can get away with this Gabe I think saw it as a stick to beat Microsoft with – and he was absolutely correct, it worked.

While VP had released a bunch of ports in the first few years after the Steam Machines initiative was announced, things have been somewhat slowing down recently. Apparently, this is nowhere limited to VP.

Peter: From the Linux angle, it is simply that publishers are either not interested, or want to place most of the risk onto the porting house. I’m not really the one to talk to about that as I don’t have much to do with the business side of it. ARMA 3 being in Beta is solely down to BI. However they are still keen on having the ports maintained and kept up to date.

As for Bohemia Interactive, their motivations seem less obvious: Keeping the ARMA3 client up to date requires some constant funding, while the Linux client remains in Beta and is not officially sold. Peter had no further information about their stance.

Peter: I’m afraid I have no info on that… but they do know what percentage of players are on Mac/Linux, as there is statistics feedback built into the game. They are not going solely by sales.

Back to the Witcher 2. On release the port wasn’t optimal. I was also testing it as it became available and the framerate was low for my hardware profile. So some criticism was due. There were however a number of heated exchanges on the support forums between VP and the community and Peter was also subject to a lot of abuse. Some of the exchanges went too far, and were more damaging than anything else.

There’s no point in fighting among ourselves when we all want the same thing: a properly working game. VP has subsequently put a lot of effort in the port and made its performance much better after a few weeks or months. What is especially ridiculous is that we keep seeing folks, in 2017 and 2018, repeating the same nonsense “Witcher 2 runs like crap”, which is completely false and a blatant lie at this point in time. The long tail of stupidity.

We tend to consider this event as an isolated Porting company/Angry gamers happening, yet its consequences reached further than what we could think. CDPR themselves were caught in the turmoil.

Peter: They (CDPR) were completely aware. Their own support staff were also getting abusive posts from the Linux community. I’m not sure if it went any further than that, but they certainly knew all about it. They also knew that I had been publicly roasted on Steam despite attempting to help resolve the problems.

Following what happened with the Witcher 2, a strong demand manifested itself among Linux gamers to secure a port for the Witcher 3, rapidly turning into one of the most requested ports on GOG’s request system. The BoilingSteam surveys demonstrated similar outcomes.

Meanwhile, Valve mentioned on multiple occasions on Steam The Witcher 3’s upcoming release for Linux, yet the port never materialized. In the end, CDPR representatives revealed the absence of such ongoing work on the port. Peter cannot confirm the actual discussions that took place at CDPR, nevertheless he conjectures numerous factors came into play.

Peter: I cannot really confirm anything solidly here, but from what I have heard it was:

1) Concerns about performance – Vulkan was not around at the time, GL was seen as a bottleneck, and Macs were clearly incapable of managing the game at the hardware level

 

2) Concerns about sales numbers. They did alright out of Mac and Linux Witcher 2, but the income was not earth shattering.

 

3) From the Linux angle, the negative and abusive response of the community, which persisted long after the initial port issues and performance were addressed. I think there were also concerns about the complexity of providing tech support for the platform.

Another unintended consequence of the Witcher 2 affair: VP were to stay silent onwards, away from prying eyes.

Peter: My boss attempted to address the concerns about Witcher 2 and got roasted for it when he made an error about the technology in his reply. Since then he has been rather unwilling to communicate with the Linux community at all. I can see his point.

At the E3 2018, CDPR revealed their upcoming Cyberpunk 2077 game. It rapidly caught the eyes of Linux gamers, eager to ask for Linux support. According to Peter, we ought nor to expect much about an upcoming port to Linux.

Peter: Personally, I think no. The Linux bridges have been burnt for CDPR. I could be wrong, but I really can’t see it happening.

A great shame if his prediction turns true.

Since Vulkan landed on the Linux platform you can almost feel the enthusiasm in the air, while very few AAA games use it presently on the Windows side of things. The promise of making multi-platform development easier has not materialized until now. Publishers or developers seem hardly invested in leading the change. For Peter this is hardly coming as a surprise.

Peter: To be honest I don’t think Vulkan is going to make a huge impact on the Windows side of things… Direct3D is simply too well known and established, and a proven technology. There are many tools e.g. for debugging built around it. There is simply no reason for change.

 

As for Macs, I think Apple simply got bored with waiting for a high performance alternative to GL from Khronos, and went off and did their own. Now they have invested in it, they won’t move to Vulkan. MoltenVK is not a “fit and forget” alternative as you still have to debug at the Metal side (and work around Apple’s driver bugs!). And Yes, Apple loves being proprietary. Some of the consoles may adopt Vulkan though. For example Nintendo Switch has it (though it is not the primary API – a custom one developed by Nintendo and Nvidia is). Maybe the next Playstation will. For Linux, I would say it’s essential as GL really has gotten long in the tooth (long past), and so it’s the only real way to compete with Direct3D performance wise.

 

Cross platform is not really a great focus in AAA – if there is enough market potential for a platform, they will develop a branch of their technology for it. For example the EGO engine Codemasters used for Dirt Showdown supported Windows, PS3, PS4, XBox 360 and Xbox One. All were separate “platforms” within the engine, and only core logic code was ever shared. Renderers were typically platform specific, even between XB360 and XB1.

 

Another thing that people are missing, is that there is still quite a lot of mileage left in D3D11, and it is a MUCH simpler API. Vulkan is extremely low level. So saying Vulkan is an alternative to Direct3D is only really true if you are talking about Direct3D 12.

As far as VP is concerned and eON, Vulkan may not be on the menu at the moment. VP progressed on Metal though.

Peter: It (eON) has Metal support on the Mac, which ARMA3 uses.

Another benefit of Vulkan is what it enables on Linux. As many others I have been enjoying DXVK to play The Witcher 3 since it won’t receive any official port.

Peter: I have not tried it. I have seen people running Witcher 3 with it, and it looks good. Given the poor performance of WINE’s attempt to do D3D10/11 with GL (it is hard !) I think it demonstrates quite well the need for Vulkan.

For Peter one of the key issues holding Linux ports back is that Linux gamers expect to get Linux clients if they have already bought the Windows version. Besides, more technical issues should be brought to attention.

Peter: One thing I think needs to change is that Linux gamers must accept that they will have to pay extra for their platform, as Mac users do. It is no good saying “I bought it and I want the Linux version later” – the porter will not get any money if you do that.

 

There are also a lot of technical issues that need to be addressed in Linux itself.. multimonitor is such a pain with X11, and Wayland is not yet ready (they need to stop fighting with Nvidia on that). Distro fragmentation DOES exist, and if you want to maximise your sales income you cannot “just target Ubuntu”.

While Peter’s argument about paying for ports makes common sense, Feral and Aspyr at least do not rely on such restrictions when it comes to Linux ports. Do they operate with different conditions?

Peter: It depends on the contract negotiated.. whether it is royalties based or flat fee based. However if a publisher thinks that they’re not going to make much money from Linux, they will push for royalties, which means all of the risk is on the porting house. I have seen a lot of Linux people claim “I bought it already, stop being greedy”. The way to think of it is that if you bought a movie on DVD, you don’t get a copy on Bluray for free later on. Likewise if you buy a PS4 game, you don’t get an XBOX One copy, even though the hardware is very similar.

 

Porters need to be paid for their work. Linux is a small market segment, and publishers are not going to just eat the cost. We’re not in a position to always demand a flat fee for the job. We can’t even demand that for the Mac, and that’s a bigger market. Fortunately Mac users generally recognize they have to pay for their platform to be supported, and are happy to buy the game again for Mac.

On the matter of fragmentation, just supporting Ubuntu may not be sufficient, and Peter recommends the following.

Peter: From the feedback I’ve seen so far, it’s pretty much Ubuntu (and derivatives), Fedora, and Arch (and derivatives like Antergos, which is what I use for testing Arch compatibility.) While I was working at VP I tested our games on Ubuntu (latest one), Mint (latest one), Fedora (latest), Antergos, SuSE and SteamOS. All on the same hardware. All of the programming was done under Ubuntu though.

SteamOS used to be a key target distro, but things have changed progressively for porters like VP.

Peter: We did initially, but soon found that we may as well just target Ubuntu. Later Valve changed to being Debian based and it really didn’t make much difference. I don’t think many hardcore Linux gamers are using SteamOS though. It is better to run a distro that is updated frequently, especially from the driver point of view. For us, Padoka’s PPA of Mesa is a godsend

Many thanks to Peter for his time and availability. It is sad to see him go away from porting games – let’s wish him great success in what he does next.

---

At BoilingSteam, we want you to browse our content free from ads and trackers. But keeping this website alive is a constant investment. Why don't you support what we do with donations on LiberaPay? Everything you contribute is re-invested in infrastructure and ongoing content to better serve the Linux Gaming community now and for future, bringing the good news to existing and upcoming Linux users. You can follow what we do via our newsletter, our RSS feed and our Mastodon profile