Running Dev Containers Locally with Podman and VSCode

Running Podman in a Docker World

Photo by Adriana Villela.

I have recently found myself in a situation whereby I needed to run Podman on my local development environment, in lieu of Docker. This was a bit of a change and a challenge for me, because I’ve been using Docker since about 2016 or so.

But hey. With great challenges come great learnings, and I am always down to learn cool stuff. So, if you’re looking to start using Podman or have started using Podman and would like a little bit of guidance, stick around!

A bit about Podman

A bit of background for the uninitiated (like me). Podman is an open source container management tool created by Redhat. It plays nice with Windows, Linux, and Mac. It has a number of commands similar to the ones that you know and love from Docker, such as podman run, podman rmi, podman images. It even has a nice little desktop manager called Podman Desktop, similar to Docker Desktop.

What it does’t have is the price tag that comes with Docker. In a nutshell: Docker is free for personal use, and smaller orgs, but you do you have to shell out the money if you’re a big org and want to use Docker for non-commercial purposes, and some companies might not want to pay that extra money if they were used to getting Docker for free in the Before Times. 💰💰💰 Totally understandable. If you’re curious, you can get more info on Docker pricing here. Podman, on the other hand, is open source and freeeeeeee.

Installing & Running Podman

Note that the instructions below apply to Mac useres. For Linux and Windows, check out the instructions here.

As a Mac user, there are 2 ways to install Podman. You can download and run the installer, or if you use Homebrew, you can run:

brew install podman
brew install --cask podman-desktop

🚨NOTE: If you go the Homebrew route, you’ll need to install Podman and Podman Desktop separately. Also, keep in mind that this is not the recommended way to install Podman on Mac, since it is community-maintained. Do so at your own risk. PS: I did it and lived to tell the tale.

Podman also has Podman Compose, which is similar to Docker Compose. This isn’t enabled by default. If you’d like to install podman compose, check out the instructions here. Or, you can install it via Homebrew:

brew install podman-compose

Once Podman has been installed, you need to create and start your Podman machine:

podman machine init
podman machine start

Why? Because under the covers it’s running a Linux VM to run your containers. Much like Docker Desktop does. The init command downloads the Podman machine image, and the start command starts it up so that you can start building and running your containers. You can’t use Podman until you run the above two commands.

As I mentioned before, Podman has its own CLI, and many of the commands are similar to what you get with Docker. If you want to avoid the mental switchover from docker run to podman run, you can spare retraining your brain with a little trick — creating shell script docker alias for podman:

sudo tee -a bleh.sh <<EOF
 #!/bin/bash
 exec podman "\$@"
EOF

sudo chmod +x /usr/local/bin/docker

NOTE: Credit where credit is due. I grabbed the above (and made a couple of changes) from this blog post.

I’ve tried this out myself, and I have to say that so far, it works pretty nicely. I haven’t hit any Docker-to-Podman CLI translation issues. It also seems to work for docker compose. Fingers crossed that it all stays that way! 🤞

Podman and VSCode

Now that Podman is installed and running, it’s time to get it set up to run with Development (Dev) Containers in VSCode. If you’re on a Mac and are setting up VSCode from scratch, you can run these commands:

# Install VSCode using Homebrew
brew install --cask visual-studio-code

# Install the VSCode Dev Containers extension
code --install-extension ms-vscode-remote.remote-containers

Once you’ve installed the Dev Containers plugin, you also need to make sure that the Dev Containers plugin is pointing to Podman, and not Docker. To do this, you’ll need to open up your VSCode settings. On the Mac, the settings can be found under $HOME/Library/Application Support/Code/User/settings.json. Check out the VSCode docs for its location on other systems.

Inside settings.json, look for the setting dev.containers.dockerPath, and set its value to “podman”, as per below:

"dev.containers.dockerPath": "podman",

If you want to get fancy, you can try to replace the existing setting using the sed command on VSCode. The command below won’t work if the setting doesn’t exist, and if you’re using an operating system other than MacOS, your location may be a little different:

sed -i -e 's/"dev.containers.dockerPath": "docker"/"dev.containers.dockerPath": "podman"/g' $HOME/Library/Application\ Support/Code/User/settings.json

Regardless of how you make the change, be sure to restart VSCode after updating settings.json.

NOTE: If you used created the docker alias for the podman command in the Installation section above, you can get away with not needing to make this change. That being said, I prefer to err on the side of caution and point to the Podman executable directly, just in case.

But we’re not done just yet, my friend! If you’re running Docker-in-Docker in your Dev Container (as I often do, so that I can run KinD in my Dev Container), you’ll also need to ensure that you add the following lines to your Dev Container JSON:

"remoteEnv": {
    "PODMAN_USERNS": "keep-id"
},
"containerUser": "vscode"

This is what the above lines look like in my completedevcontainer.json files:

For more info on using the VSCode Dev Containers plugin with Podman, check out the VSCode docs. I also found this GitHub issue helpful and used the configuration snippet from there.

Gotchas

Before we wrap things up I’d like to share a few gotchas that I encountered on my Podman with VSCode journey.

Resource Allocation

I have a really really really beefy machine, and I was really scratching my head over the fact that when I first tried to run my Dev Containers a) my Dev Container images were taking FOREVER to build and b) once inside the Dev Container, things were suuuuuper slow.

I finally realized the culprit: I hadn’t allocated sufficient machine resources to Podman. In fact, I hadn’t edited my resource allocations at all since I installed Podman. The main culprit was memory allocation: only 2GB RAM had been allocated to Podman, which was a very very very small percentage of the memory that I had available to allocate, and also woefully inadequate in general. You can also tweak the CPU and Disk Size Settings.

To view your settings go to Settings > Resources in Podman Desktop:

Podman Desktop Resources configuration screan

Or, if you’re like me and want to know where the configuration files are, you can find this info in the podman-machine-default.json file located in $HOME/.config/containers/podman/machine/applehv on Mac. More information on podman-machine-default.json and its location on other systems can be found here.

The resources configuration section looks like this:

"Name": "podman-machine-default",
"Resources": {
    "CPUs": 7,
    "DiskSize": 100,
    "Memory": 43869,
    "USBs": []
},

Which, as you can see, matches up with what I have in the Podman Desktop screenshot above.

Extension Host Terminated Unexpectedly

After I finally got my Dev Containers up and running, I kept periodically getting the following pop-up message in VSCode from within my Dev Container: Extension host terminated unexpectedly. Which meant that my Dev Container environment kept cutting out all the time and I couldn’t get any actual work done. Very frustrating.

I eventually made the error go away, but I don’t know if it was related to two culprits or one, because it was also happening at around the time that I noticed my woefully inadequate resource allocations for Podman. So bumping my resources may have fixed it.

But there’s also another possibility, which was described in this StackOverflow post: there may have been a VSCode extension (or two or three or…) that was causing problems too. The easiest course of action is to just disable each extension one by one (except the Dev Container one!!) until the the problem appears to be resolved. The other option, as one user suggested, is to run the Extension Bisect Utility, which helps speed up the extension troubleshooting process.

My Dev Container was failing to build!!

This one hit me just this morning. I had built a Dev Container last week. It was working fine. I thought that I had made no changes since I’d built it last week. (Ha! Famous last words!) Then I rebuilt it from scratch today, just to make sure that things were still working smoothly, and then…I ended up this nasty error saying that VSCode couldn’t build my Dev Container because it was failing to install the Docker in Docker feature.

Feature "Docker (Docker-in-Docker)" (ghcr.io/devcontainers/features/docker-in-docker) failed to install! Look at the documentation at https://github.com/devcontainers/features/tree/main/src/docker-in-docker for help troubleshooting this error.

I’ll be honest — I tried a few things, because I was bordering on desperation and frustration so early on a Friday morning.

1- Pruned my volumes

podman system prune --volumes -af

2- Removed dangling images

podman image prune -f

3- Restarted Podman

podman machine stop
podman machine start

And when all else failed, I rebooted my computer.

And while I think that each of those things was helpful in their own right, after playing around a bunch, I think I’ve narrowed the issue to how I initially defined my docker-in-docker feature in my devcontainer.json:

"ghcr.io/devcontainers/features/docker-in-docker:2": {
    "version": "2.12.0",
}

I thought I was being smart by locking down the version. Leaving out a version altogether is equivalent to setting “version”: “latest”, and that’s generally not a good practice. But alas, setting the version is exactly what was causing me problems.

So I just went with the feature configuration below, and that did the trick:

"ghcr.io/devcontainers/features/docker-in-docker:2": {
    "version": "latest",
}

To be sure that there was no additional funky business, I made sure to build my Dev Container in VSCode using the Dev Containers: Rebuild Without Cache and Reopen in Container” option:

Rebuild Dev Containers Without Cache option in VSCode

I suspect that if you encounter similar issues with other Dev Container features, try leaving out the feature’s version to see if that helps fix things.

Final Thoughts

Moving from Docker to Podman was overall a pretty good experience once you get over some of those initial hiccups and gotchas. Some of the gotchas have to do with Dev Containers themselves — not related at all to Podman or Docker, which always adds to the joy. The good news is that you’ve got some info at your fingertips now, so hopefully your transition from Docker to Podman is much smoother than mine.

This experience has definitely made a convert out of me, and a week into using Podman, I’d say that I got comfortable with it pretty quickly. Not too shabby!

With that, I will leave you with a photo of my more elusive rat Buffy (the Vampire Slayer), getting cuddles from my daughter.

Buffy comes out for a rare photo op.

Until next time, peace, love, and code! ✌️💜👩‍💻