Frederic Charette

I truly enjoy minimalism in life. Clutter makes me anxious, it overloads my brain. I try to follow the inbox-zero strategy. I don’t own many things and avoid subscriptions like the plague. This is also reflected in how I manage my work environments, fewer gadgets, fewer levers, fewer levels of abstraction and no “magic”. Just simple ingredients cooked well.

And I do feel vindicated when I see the current landscape in cybersecurity. Right now, high-visibility tools and popular npm packages are targeted by supply chain attacks. Critical tools, like FFMPEG and VMware products, continue to receive vulnerability disclosures. Not to mention the role of AI in accelerating all of this. Through simple logic, I like to think that in reducing moving parts, I reduce my surface and thus potential attack vectors.

A good opportunity for minimalist design presented itself to me recently. I’ve been playing with local LLMs on my homelab running Proxmox. Simple models that can run on CPU to perform menial tasks. By the way, check out the excellent ik_llama.cpp project if you’re thinking of doing something similar!

In order to get decent performance, I store the model in RAM ( — mlock). As you can imagine, this container on my homelab eats up a sizeable chunk of the available memory. And because hardware isn’t cheap and I don’t wan’t to buy more, I thought: Why wouldn’t I shut down my service when I’m not using it instead? Ideally, in a non-manual way… but still simpler than bringing in Kubernetes or some other enterprise-sized monstrosity.

I started my search in the most naive way, not knowing that I was already sitting on an incredibly powerful tool and only using it to pass the butter.

We’ve all used systemd to create services to run commands on boot… but I, for one didn’t know how capable and complete it really is. Let’s break down my unit file from the simple “run on boot” setup to a complete wrapper that:

• Limits what my unit can do and what folders it can access
• Fully shuts down when there is no traffic
• Boots up when network requests come in
• Implements a reverse proxy
• Manages credentials, secrets and keys

No 3rd party software, no extra control plane, just software that ships by default on most Linux distros!

“The basic”

Tutorials on the web will usually stop at setting up the most basic unit so it starts a program on system boot. Here’s our ikllama.service:

[Unit]
Description=My llm service
Wants=network-online.target
After=network-online.target

[Service]
ExecStart=/root/ik_llama.cpp/build/bin/llama-server --model /root/models/Ministral-3-8B-Reasoning-2512-Q5_K_M.gguf -ngl 0 -rtr --no-mmap --mlock --ctx-size 65536
Restart=on-failure
RestartSec=3s

Here we’ve defined a unit that will be executed once the network is online.

You’ll notice already that this isn’t the most secure configuration. Indeed, you should avoid running services as root!

Let’s create an unprivileged user, then move the executable and the models to that user’s home. We’ll update our service with the new user info and also add a few more security sandboxing options.

[Unit]
Description=My llm service
Wants=network-online.target
After=network-online.target

[Service]
User=ikllama

KeyringMode=private
ProtectClock=yes
ProtectHostname=yes
ProtectKernel=yes
NoNewPrivileges=yes
PrivateDevices=yes
ProtectSystem=full

ExecStart=/home/ikllama/ik_llama.cpp/build/bin/llama-server --model /home/ikllama/models/Ministral-3-8B-Reasoning-2512-Q5_K_M.gguf -ngl 0 -rtr --no-mmap --mlock --ctx-size 65536
Restart=on-failure
RestartSec=3s

# Prevents access to the keyring of users
KeyringMode=private

# Prevents changes to the system clock
ProtectClock=yes

# Prevents changes to the hostname or NIS domain name of the system
ProtectHostname=yes

# Prevents explicit loading of kernel modules
ProtectKernelModules=yes

# Prevents processes from gaining new privileges
NoNewPrivileges=yes

# Limits access to pseudo-devices such as /dev/null, /dev/random, /dev/urandom, and /dev/zero
PrivateDevices=yes

# Marks some file system paths as read-only (ex: /usr, /boot, /efi, /etc)
ProtectSystem=full

You may choose to add or remove some service hardening rules since your mileage may vary depending on the application and the specific setting. When an app is untrusted, you can specify explicit ReadWritePaths that a service has access to and even lock down the processes’ CapabilityBoundingSet.

In doubt, a number of tools exist to test whether an app requires particular system calls or capabilities. systemd-analyze security, strace or getpcaps just to name a few.

Let’s scale

This is where things get exciting. Instead of starting the service at boot, let’s use socket activation to start it when a connection arrives. We’ll need to create a new service, which we’ll name ikllama-proxy.socket

[Unit]
Description=ikllama Socket Activation

[Socket]
ListenStream=80

[Install]
WantedBy=sockets.target

Two things to notice here, one is that we’re listening on port 80- which normally requires some configuration since this is a privileged port. But here systemd will act as our reverse proxy and has no issues with this!

Second is that our ikllama doesn’t bind to port 80! Which means that we need to make another systemd configuration to create our simple reverse proxy leveraging systemd-socket-proxyd. We’ll name it ikllama-proxy.service

[Unit]
Description=ikllama Socket Proxy
Requires=ikllama.service
After=ikllama.service

[Service]
DynamicUser=yes
PrivateDevices=yes
ProtectHome=yes
ProtectKernelTunables=yes
ProtectKernelModules=yes
ProtectKernelLogs=yes
ProtectControlGroups=yes
ProtectClock=yes
ProtectHostname=yes

ExecStartPre=/bin/bash -c 'for i in \$(seq 1 150); do (echo > /dev/tcp/127.0.0.1/8080) >/dev/null 2>&1 && exit 0; sleep 0.2; done; echo "ikllama backend on port 8080 did not become ready in time" >&2; exit 1'
ExecStart=/lib/systemd/systemd-socket-proxyd 0.0.0.0:8080

Next we’ll need to modify the unit section of our ikllama.service

BindsTo=ikllama-proxy.socket
After=ikllama-proxy.socket

Now when we visit 0.0.0.0:80 our service starts and we get served our app.

Note that it is also still available at port 8080. Visiting that address does not trigger the service startup though. We may also want to create a firewall rule to restrict access to that port.

It is important that your proxy service and socket have the same name so the socket reference is passed.

What about shutting down

Unfortunately, the strategy I came up with to shut down idle services is a bit less polished than previous steps.

We’ll create a new service called ikllama-monitor.service

[Unit]
Description=Network Idle Monitor for ikllama
After=network.target

[Service]
Type=simple

NoNewPrivileges=yes
PrivateTmp=yes
PrivateDevices=yes
ProtectSystem=full
ProtectKernelTunables=yes
ProtectKernelModules=yes
ProtectKernelLogs=yes
ProtectControlGroups=yes
ProtectClock=yes
ProtectHostname=yes

ExecStart=/root/scripts/monitor.sh 80 ikllama
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target

This service launches a bash script (available on github) that periodically checks for activity on a given port. If no activity is detected, the ikllama service is stopped.

Now, when I visit my llm’s webgui, the service starts (almost instantly). And a little while after I close the tab, the RAM is reclaimed in my Proxmox environment.

To try all of this for yourselves I have begun compiling a set of tools for homelab enthousiast such as myself here: https://github.com/Nerve-box/dotfiles

Contributions and suggestions are welcome!

What this means for homelab security

By reducing the control plane to purely the OS (systemd, firewall), we reduce the number of software components that need to be trusted, configured and kept patched.

Furthermore, it forces us to better understand how software runs in a given environment and to properly sandbox it, developing good security reflexes.

I hope you learned a thing or two. I sure did!

Happy homelabbing!

References

https://linux-audit.com/systemd/settings/units/

https://www.freedesktop.org/software/systemd/man/latest/systemd-socket-proxyd.html

How I used AI to increase my library’s performance by 5x !

A tale of using AI to highlight your blind spots.

SPOILER: AI didn’t deliver the performance improvement itself, but it got me looking at the right places. Hey, I know, clickbait title- but while you’re here why not stay and enjoy my tale.

In the beginning…

I was looking for a new project to work on in my spare time. I have always loved toying with the HTML canvas and sockets, sharing them on JS1K, JS13kGames and others. I got really inspired by a recent project by Nolan Royalty: One Million Chessboards.

I set out to build a similarly large-scale online web game, and I knew I had the perfect tool in my belt: Kalm, a socket management library I had built a decade ago to facilitate congestion control in real-time applications. But I decided to also give myself another challenge for this project… co-develop it with this junior developer called “Claude”. (Gotta give this “vibe-coding” stuff a try, apparently it’s all the craze)

The (first) project…

Claude could understand basic instructions pretty well, and was very polite- always asking before performing any commands. Though however, he would often get lost whenever a CLI program required extra input.

I ended up setting the project structure myself, just so it wouldn’t need to improvise too much. This limitation would become very clear in future steps.

I now had a basic Svelte app. Claude had given me a canvas element, a render loop and a few other basic functionalities. It was now time to build a socket server, so I gave Claude a prompt that, even myself, found perhaps too daring: “Build a node.js websocket server that uses the Kalm framework”.

Surprisingly, it delivered… kinda. There was now a folder with the node.js server. It had installed Kalm as a dependency and there was code that- at first glance- looked promising. I gave it a spin.

Immediately, I was met with strange errors. A few were my junior’s fault, not keeping track of certain variables, hallucinating others… but something wasn’t right. Namely, Claude had installed an *ancient* version of my library.

Understanding AI…

The code was modern, but the version installed was old?

Without thinking too much about it, I prompted Claude to “use the latest version of Kalm”. It apologized profusely (not really, but it’s fun to imagine) and installed the latest version available at the time: v7.0.0.

The code remained unchanged, but now the app compiled.

Why did it install this old version in the first place?

• Was my repo still publishing some old examples?
• Did it pick up on an article on “how to use kalm” ?

This all got me thinking… if AI gets confused, maybe devs (and their AI co-pilots) could get confused as well!

I panicked and cloned my library’s repo to end this madness at once!

The second project���

Right off the bat, it was clear I needed to update a few things, as most OSS developers painstakingly do on a regular basis.

• Update LICENSE date
• Bump engine requirements as Node versions go end-of-life
• Bump dependency versions
• Update toolchain when packages decide to deprecate certain functionalities
• Re-test everything

Once those tasks were done, I had new missions:

• Add code comments in areas where the logic isn’t immediately obvious
• Add more code examples with detailed READMEs
• Make sure that old examples and references are updated

These steps were what kicked off one of the largest overhauls my library had seen in many, many years. Standardizing variables names, documenting flows and refactoring when unclear. Building many example apps, highlighting bugs and inconsistencies. Not least of which two bugs that caused a major performance degradation!

I was happy… but not quite done. I still hadn’t found out why it installed that old version.

That was, until I went to fix a typo on the library’s static website page and noticed this:

Do you see it?

This static page, last updated 7 years ago, was sending everyone to an ancient version of kalm’s example folder.

A majestic facepalm ensued.

In conclusion…

AI co-pilots are very much like junior developers in that they lack instincts and reflexes. They simply follow links and obey.

If you are an app or OSS developer and you want your projects to be approachable by both organic and non-organic lifeforms, make everything dead clear and update your docs :)

Oh, and check out Kalm!

I recently read a nice article from former Navy Seal Brent Gleeson about Ways to lead by example. It really inspired me to apply these key principles to the management of software development teams. I was surprised by how the structures of teamwork applied so neatly to the world of Software development. Here are the 7 tips on how to inspire your team and be a great Engineering manager.

Get your hands dirty. A good leader must understand the struggles of the development craft and have intimate knowledge of the company’s services and systems. Without proper experience, it becomes very difficult to make informed decisions. Personally, I’ve tried picking a few technical tasks within the teams that I manage to make sure that I stay relevant. I’m also forcing myself to learn about areas I’m less familiar with so that I can navigate our projects with ease.

Watch what you say. Making irresponsible promises, complaining or fooling around may pass when you have no one depending on you, but that changes drastically after you are given direct reports. You may hurt morale, undermine leadership or worse: cause an employee to leave. Whenever I have to talk I try to remember these three rules on what I’m about to say:

• Is it true
• Is it kind
• Is it necessary

Respect the chain of command. Like the original article says, nothing creates confusion like breaking the rules of hierarchy. As a leader, you set the tone for how you want your team to act with others. If you go around people or processes to achieve your goals, don’t get surprised if one day your team goes around you in the same way.

Listen to the team. A lot of software development managers come from a technical background as devs. For most of us, it’s fostered a bad habit to want to fix anything remotely out of place, thinking it’ll make the team better. Take some time to listen to your devs, their blockers and their ideas instead of projecting your own concerns. They currently live in the code and know it best.

Take responsibility. When incidents happen, or when someone drops the ball, we all collectively feel the blow, but leaders should feel the most responsible for the mistakes as they should’ve better prepared their teams. Whether by encouraging more rigorous deployment practices, testing standards or even by improving task estimation. Use these opportunities to strengthen your team instead of throwing one of your own under the bus.

Let the team do their thing. We hire professionals, we spend many hours grilling them in interviews to make sure that they are not only capable, but that we would actually trust them with our project. Maybe it’s time we act like that means something.

Don’t hesitate to remove yourself from meetings, trust your devs to do the right thing in your absence. You’ll have more time to focus on other problems.

Take care of yourself. I can’t stress that one enough. Leaders don’t just inspire by their work ethics and technical skills, they also inspire on a personal level. If the only image of you that your team sees is someone who lives for the job and works overtime, they will feel the need to behave in the same way.

If instead you control your work hours and share some of your personal hobbies, you will inspire them to do the same. A happy and rested team will always perform better, and churn less!

Leading by example, applied to Engineering teams was originally published in Pleo Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

For most people, being interviewed is not fun. There’s the pressure of giving a good first impression, performance anxiety along with the uncomfortable feeling of being observed and judged.

Employees don’t necessarily enjoy running interviews either. It’s outside the comfort zone of many, it drives away from scheduled work and they may not even know what to look for in a candidate.

There isn’t a simple magic recipe for conducting interviews. But if I had to pick a guiding principle, it would be that the experience be inspiring for both parties. Candidates need to leave the interview more excited about the company than when they entered.

I’ve developed a technique over the last 10 years with that principle in mind. It is divided into 4 sections.

1. Building trust

Your first responsibility as an interviewer is to build trust with the candidate. A study showed that the stress of being interviewed is enough to reduce performance by 50%. And while it may not impact everyone in the same way, stress will skew the results of your assessment.

How can you, as an interviewer, build trust and potentially reduce stress? Think of how you would act if the person that you are interviewing is an old childhood friend you haven’t seen in years. You may not know who that person is anymore, but you are eager to find out.

Open with a warm smile, a handshake (if it’s currently safe to do so), inquire if the directions to the office were sufficient, if they need to go to the bathroom, or if they want something to eat or drink.

After everyone is set, I like to take a page from the 5 Dysfunctions of a team: Be vulnerable, share something from your past, something personal. It could be how you got into programming, an anecdote about your nickname or something similar. This is a great way to break the ice during introductions and often creates a visual imagery to help candidates associate with the interviewers and can also help them to remember your names more easily.

Trust building does not stop at that point, though. Trust is built in drops, but lost in buckets. Staying aware of your tone and your body language throughout the interview can help you preserve this bond.

2. Setting expectations

The second aspect of my approach is to set clear expectations for everyone, starting before even posting the job offer online. What is your team looking for in terms of immediate competencies? Where do you see a potential new hire in 1… 2… or even 5 years? Do you expect them to be in the same role, to grow into a domain expert, a manager, etc.

This projection of a candidate’s growth inside a company, or “upside”, is crucial in finding candidates that will grow into key people in the organization. They should be motivated by the company’s mission and its challenges.

Stepping into an interview, the candidate should have a clear idea of what the mission is, and what the company’s challenges are. You can take a minute to assess this at the beginning of the interview. What skills are required in the short term, what is expected of a person in that role after X months or X years with the company. Check to see if there’s alignment with their own career goals.

Be honest and transparent. Ask questions about what is important for the person in the work environment and try to highlight aspects of the company that might be most relevant to them.

3. Leading the conversation

Interviews can be awkward with interviewers asking narrow questions and expecting neat answers. That’s not usually how people interact in real life.

To combat this, try to frame the question with some premise, or some context. For example:

“Here at Pleo, we make good use of kafka, it’s how we tie most of our services. I see that you have some experience with it, can you think of a way to re-consume events?”

This way, the candidate has a better idea of where the question is coming from, making the conversation sound more like well… a conversation. Meaning that if you want to ask follow up questions based on a candidate’s answer, it won’t stick out too much and put them on guard.

And you should absolutely ask follow-up questions. They help in confirming whether or not the person just memorized the wiki page on a topic, or if they have a meaningful understanding of it. Some of my favorites include:

• Do you know of similar technologies, processes or alternative ways to address the problem?
• Can you think of situations where you should or should not use a given pattern or technology.

Beyond technical competencies, you may also want to ask follow up questions to evaluate soft skills, such as Curiosity, Collaboration and Humility. Questions like:

• Are you familiar with how this technology works, under the hood, or why it came to be developed
• How would you break down a project for you and your team, implementing this technology
• Do you have any war stories about working with that technology and what did you learn from it?

Keep your questions not overly specific, remember that you want to evaluate the communication, humility and passion of candidates, not just their technical skills.

4. Whiteboarding

There are a lot of critiques of whiteboard interviews and most of them are valid. You will never have to code on a whiteboard or be asked to solve a problem without being able to use StackOverflow.

That being said, whiteboarding is something that we do every day as part of our work. We have dry-erase boards in meeting rooms and sometimes entire walls or windows, we go through large stacks of lined paper, post-its, etc! The difference is that we don’t use these tools to code, we use them to explain. Thus, their communication skills are exactly what we should evaluate when candidates are on the whiteboard.

Give them a problem to solve, tell them that they can use whatever language or syntax they want. The intention here is that you simply want them to run through the problem in as much detail as possible.

Give them 10–15 minutes to do the exercises, reminding them to be communicative all through the exercise and to explain the problem, edge cases and potential solutions with the interviewers.

Conclusion

With all of these tools, you should enter interviews with more confidence and notice a positive impact on how candidates behave and succeed throughout the process. You may want to monitor online reviews, like Glassdoor to see how your interview process is perceived and what areas you should work on.

One last thing: Take comprehensive notes on the interview for your peers to review after the fact. Make sure to not only include technical aspects and responses, but also soft skills and attitude of the candidate. This can be useful to identify which areas to cover in future interviews.

The interview tangle was originally published in Pleo Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

It’s no secret that node_modules, while a wonderful solution to local package management, is also regarded as a swarming nightmare of files, which create a lot of pressure on disk IO and space.

This can be very problematic for smaller devices running node as is now more popular in IoT.

There are a couple of neat solutions to bundle all of your dependencies in a single js file, like yarn’s PnP proposal explores. Unfortunately, this doesn’t necessarily work for IoT devices with low computing power as parsing such a large JS file would lock everything for a few seconds, or perhaps longer. This concept has been well explored in this fantastic article: The cost of javascript in 2018

Instead, I tried understanding *what* exactly can be found in your node_modules. The results may shock you…

- Benchmarking suites
- Wikis and documentation (with image/video assets)
- Test suites
- Internal tooling
- Raw sources (alongside transpiled modules)
- Code samples
- Guy Fieri ?

Right now, as you read this, these files are sitting inches away from your code, on your production environments.

NPM allows you to dry-run package deploys to see what files are going to be included. This is a step often skipped by package maintainers. NPM currently does not warn you about the content of your package, and `npm install — prod` doesn’t help with this problem.

So, what’s the solution ?

I created a small set of instructions that I added to a dockerfile that finds and deletes most known offenders. It could be made into an npm package itself (ironic if you ask me).

Here it is: https://gist.github.com/fed135/af42be63989e734acc025090fd120186

Let me know if I missed any.

We had a project for work at one of my last jobs to create a replica infrastructure in Europe, our main one being in Canada (for legal reasons). During the conversation, there was a question as to whether we would need to replicate everything, or if the European services could leverage some of the Canadian ones.

Of course that would allow us to save a pretty penny on EC2 costs, but what about the end user performance impact of making requests that need to physically travel back and forth the Atlantic Ocean.

I embarked on the (quick) journey to build a tool that would estimate the minimum amount of time (ms) required for a packet to make that trip, based on physics.

The premise is that a packet of information may not travel faster than the speed of light. This premise is backed by the fact that the speed of light is the fastest speed at which energy, matter and information can travel. (source)

With this constant, along with the geographical positions of AWS data-centers, plus some maths for the actual distance on the globe (the earth is a sphere, after all), we are left with the theoretical minimal latency that can be expected for a packet to travel from one region to the other.

Now, the next step was to get a rough idea of a full round trip (or ping) time in ms. For this, I had to read into this paper of network hops and the added latency that is expected: (source)

Once put together, I was able to get the minimal round trip overhead from Europe (eu-west-1) to Canada (ca-central-1)

Ping: 53.89 ms

…and now comparing with actual numbers

Ping: 78.75 ms

It’s not exactly accurate, but it can give us a lot of insights:

• We can plan SLAs more accurately for routes needing to reach other AWS regions,
  - We can estimate infrastructure overhead and potentially detect outages (if we take the full round trip time from the European service’s request, then subtract the minimal ping value and the Canadian service’s response time, we’re left with the infrastructure overhead)

Here’s the tool for those interested: https://gist.github.com/fed135/fdac7af0a49099ad45cc2547a45bf762

Anyways, that’s all I had for today.
Cheers