Continuing with the subject on kubernetes debug containers, let’s imagine a scenario where you need to change a configuration or file within a minimal app, so no terminal or any other tools.
As this is minimal container, you won’t be able to exec -ti -- bash into the container to push for any changes.
I’ve already described briefly debug containers at my previous post, and now I’ll describe a few options that were used on that post but not fully explained.
Pod Setup
First let’s setup a minimal nginx pod.
Let’s use nginx, as it has a index.html default page that we can modify on the fly and check it’s content.
I’m going to use the images generated by jbeverid - Joseph Beveridge: nginx-distroless.
So the pod to run our nginx-distroless we can run the following:
kubectl run nginx-minimal --image=jbeveridge/nginx-distroless:1.27.2 --restart=Never --dry-run=client -o yaml
apiVersion: v1
kind: Pod
metadata:
labels:
run: nginx-minimal
name: nginx-minimal
spec:
containers:
- image: jbeveridge/nginx-distroless:1.27.2
name: nginx-minimal
resources: {}
dnsPolicy: ClusterFirst
restartPolicy: Never
status: {}
It seems to be all good with the yaml, so we can either just run the command above without the --dry-run and -o yaml options or save the yaml and apply/create it.
I’ll run the imperative command directly to create the pod:
kubectl run nginx-minimal --image=jbeveridge/nginx-distroless:1.27.2 --restart=Never
After the pod is running you can check the logs, but as it’s a minimal nginx even the logs are minimal, but let’s try to do some requests to the / (root main index.html page), and let’s check the logs.
Before running any requests let’s port-forward the pod to our local environment, so we can run local curl requests.
k port-forward pods/nginx-minimal 8080
Forwarding from 127.0.0.1:8080 -> 8080
Forwarding from [::1]:8080 -> 8080
Handling connection for 8080
According to the image docs nginx is using port 8080, so I’m just port forwarding it. If you have a conflict, just map it like 8080:8081 to any different port you might need.
Finally we can test a curl request:
curl -X GET http://localhost:8080/
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
And we see as a result the default nginx index.html page.
Using the ephemeral debug container to push changes
If we try to -it (TTY) it won’t work, as there’s no shell, nor package manager in the image. So at this point the container is locked in terms of shell access, privileges and changes, and thats very good in terms of security.
We stil have ways to access the image by using debug containers. My previous post on the same subject uses a similar example, but it’s not explaining certain flags and options.
To achieve what we want on such use cases we have to mainly use two flags with the debug container: --target and --profile.
-
--target: targets the process namespace of another container. It will share the namespace of the target container with our debug environment, making it possible for us to see the process, plus access it’s filesystem below/proc/<PID>/*. -
--profile: applies a pre-defined profile on attachng to the target container, so that we can have root privileges, what in our case is not needed. Privileged access on k8s is to give access or CAPs to the container on the host node, and our changes here are mainly scoped to container changes. We will use thegeneralprofile though, as it withlegacywe ended up with no access to certain parts of the namespace.
I’m going to use an ubuntu, image just to have a full set of package manager, terminal and others.
kubectl debug -it nginx-minimal --image=ubuntu:24.04 --target=nginx-minimal
kubectl debug -it nginx-minimal --image=ubuntu:24.04 --target=nginx-minimal
Targeting container "nginx-minimal". If you don't see processes from this container it may be because the container runtime doesn't support this feature.
--profile=legacy is deprecated and will be removed in the future. It is recommended to explicitly specify a profile, for example "--profile=general".
Defaulting debug container name to debugger-tthnv.
All commands and output from this session will be recorded in container logs, including credentials and sensitive information passed through the command prompt.
If you don't see a command prompt, try pressing enter.
root@nginx-minimal:/#
Let’s see if we can see the processes running or shared with the same namespace as the debugger container:
root@nginx-minimal:/# ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
10001 1 0.0 0.1 9752 6244 ? Ss 08:07 0:00 nginx: master process nginx -g daemon off;
10001 7 0.0 0.0 9996 2308 ? S 08:07 0:00 nginx: worker process
root 8 0.0 0.0 4300 3488 pts/0 Ss 08:44 0:00 /bin/bash
root 16 0.0 0.0 7632 3468 pts/0 R+ 08:46 0:00 ps aux
The nginx-minimal image is prepared to run with a non-root user 10001, and then we can see the processes running, PID 1, and PID 7 for workers.
Let’s check the filesystem below /proc for the PID 1:
root@nginx-minimal:/# cd /proc/1/root/
bash: cd: /proc/1/root/: Permission denied
It seems that with the profile legacy, the default, is causing some issues on accessing properties on the main container namespace.
As it’s a profile being deprecated, let’s laeve this debug container and start a new one with the general profile.
kubectl debug -it nginx-minimal --image=ubuntu:24.04 --target=nginx-minimal --profile=general
Targeting container "nginx-minimal". If you don't see processes from this container it may be because the container runtime doesn't support this feature.
Defaulting debug container name to debugger-lgnpv.
All commands and output from this session will be recorded in container logs, including credentials and sensitive information passed through the command prompt.
If you don't see a command prompt, try pressing enter.
We can still see the processes but let’s see if we can actually access the /proc/1/root/ dir properly:
root@nginx-minimal:/# ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
10001 1 0.0 0.1 9752 6244 ? Ss 08:07 0:00 nginx: master process nginx -g daemon off;
10001 7 0.0 0.0 9996 2308 ? S 08:07 0:00 nginx: worker process
root 28 0.1 0.0 4300 3488 pts/0 Ss 08:54 0:00 /bin/bash
root 36 0.0 0.0 7632 3532 pts/0 R+ 08:54 0:00 ps aux
root@nginx-minimal:/# cd /proc/1/root/
bin/ boot/ dev/ etc/ home/ lib/ proc/ root/ run/ sbin/ sys/ tmp/ usr/ var/
root@nginx-minimal:/# cd /proc/1/root/
root@nginx-minimal:/proc/1/root#
root@nginx-minimal:/proc/1/root#
Yes we do have access now to the directory, let me try to fing any index.html file:
root@nginx-minimal:/proc/1/root# find . -name 'index.html' 2>/dev/null
./etc/nginx/html/index.html
./usr/share/nginx/html/index.html
Found 2 files for nginx default index.html page, let me try to modify the first one and try a new request to see if we have any changes.
We will need as well to reload this page on nginx through another exec to the container.
As I’m using ubuntu, with a full package manager, I’m going to install vim to do the changes, so you can run the following as well if you’re using ubuntu to install it: apt-get update; apt-get install vim.
So here’s the modified page:
<!DOCTYPE html>
<html>
<head>
<title>Welcome to NGINX!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to NGINX!</h1>
<p>If you see this page, the NGINX web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using NGINX.</em></p>
</body>
</html>
I’ve changed some mentions of nginx to NGINX on caps.
Let’s run a command on our nginx-minimal pod/container to reload it:
k exec nginx-minimal -c nginx-minimal -- nginx -s reload
And the result is:
k exec nginx-minimal -c nginx-minimal -- nginx -s reload
2026/06/24 09:07:14 [notice] 383#383: signal process started
Let’s do another curl call and check the content back:
➜ ~ curl -X GET http://localhost:8080/
<!DOCTYPE html>
<html>
<head>
<title>Welcome to NGINX!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to NGINX!</h1>
<p>If you see this page, the NGINX web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using NGINX.</em></p>
</body>
</html>
We can see some of the nginx as NGINX (all minus the url links).
We had to change --profile from the default legacy to general to achieve our goal because general is more permissive in terms of access and permission for the nanmespace, so being more suitable for debugging tasks nowadays. Legacy might not work properly, so not giving your enough permissions to changes within the container. Leep in mind that sysadmin is not always needed, and it opens access to the host node permissions, and the same for netadmin. So be careful on using those privileged containers when doing debug, and use the proper profile according to the task needed and permissions.
Best Practices for configs on k8s and or small static content
The case described above is just an example, and it can be use on emergency situations where changes have to applied in order to have your app runnning, or on tests for example where you want to change things on the fly.
The best practices though for k8s is to use ConfigMaps and mount, use as env vars or whatever way you need, but on a controlled way and put it on version control.
ConfigMap Example with some long config:
apiVersion: v1
kind: ConfigMap
metadata:
name: test
data:
config.yaml: |
config:
someketconfig:
- name: config1
value: value1
- name: config2
value: value2
Here’s how the config looks when described:
➜ ~ k describe cm test
Name: test
Namespace: default
Labels: <none>
Annotations: <none>
Data
====
config.yaml:
----
config:
someketconfig:
- name: config1
value: value1
- name: config2
value: value2
BinaryData
====
Events: <none>
With the notation used above we could even have the index.html page from the test as some sort of static content to be pushed:
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-main-page
data:
index.html: |
<!DOCTYPE html>
<html>
<head>
<title>Welcome to NGINX!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to NGINX!</h1>
<p>If you see this page, the NGINX web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using NGINX.</em></p>
</body>
</html>
Let’s describe it:
k describe cm nginx-main-page
Name: nginx-main-page
Namespace: default
Labels: <none>
Annotations: <none>
Data
====
index.html:
----
<!DOCTYPE html>
<html>
<head>
<title>Welcome to NGINX!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to NGINX!</h1>
<p>If you see this page, the NGINX web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using NGINX.</em></p>
</body>
</html>
BinaryData
====
Events: <none>
And just as a simple example lets mount the configmap on a pod:
apiVersion: v1
kind: Pod
metadata:
labels:
run: test-cm
name: test-cm
spec:
containers:
- args:
- sleep
- "3600"
image: ubuntu
name: test-cm
resources: {}
volumeMounts:
- name: test-cm
mountPath: /app/config
dnsPolicy: ClusterFirst
restartPolicy: Always
volumes:
- name: test-cm
configMap:
name: test
status: {}
Checking the config.yaml file mounted in the path:
➜ ~ k exec -ti test-cm -- bash
root@test-cm:/# ls -ltr /app/
total 0
drwxrwxrwx+3 root root 79 Jun 23 09:01 config
root@test-cm:/# ls -ltr /app/config/
..2026_06_23_09_01_19.1035488612/ ..data/ config.yaml
root@test-cm:/# cat /app/config/config.yaml
config:
someketconfig:
- name: config1
value: value1
- name: config2
value: value2
root@test-cm:/#
There’s more on Configmaps on the kubernetes official documentation like using items to even filter certain keys to mount or push as env vars and more.
PS: As describe at the title, small static content can be pushed as a configMap, but consider as well pushing it to the image if the content is still static but it wouldn’t be easy to maintain in a configMap, like longer content/configs. Of course if your content is dynamic try to push it as part of a deployment / or adding content in a different way, or part of a build process for example. Another thing to consider is that this content might not be subjected to linters and code checks, what can cause outages, so prefer to push some of the content as a deployment/git repo, and you make use of a sidecar container, that takes care of the gitsync, in order to pull the content to the main pod / container, as they can share an emptyDir mount.