Our embedded products run on Zephyr and Nordic Semiconductor's nRF Connect SDK. These libraries are written by others, but we ship them in our products — so when they fail, it's our job to understand why. Here's a good example.
One of our LTE-M gateways, built around an nRF9151, worked perfectly in the lab. Then a batch of SIM cards arrived, and some of them simply refused to connect to our application server. Same board, same firmware, same APN, same server. Only the SIM card changed.
The investigation took us through modem AT traces, the EPS session management layer, M2M remote SIM provisioning, and finally into the code of the SDK we use every day. It ended with a fix that is now part of the nRF Connect SDK, for every developer who uses it.
The symptom: a bug that follows the SIM card
The gateway opens a secure DTLS session (CoAP) to our server as soon as the cellular network is up. With some SIM cards, it failed every time: the handshake struggled, then the very first data packet sent was rejected.
The first reflex, when a product behaves differently from one unit to the next, is to suspect the hardware. So we swapped SIM cards between gateways. The verdict was clear: the problem follows the SIM card, not the board. Four cards in the batch showed it, systematically. A fifth, from the same plan and on the same APN, worked flawlessly.
At that point, it's tempting to conclude "bad SIM card" and send the batch back. We preferred to understand.
Reading the modem's AT traces
The nRF9151 modem reports its state through AT notifications. By placing the trace of an affected card next to that of the reference card, the difference jumps out within a few hundred milliseconds.
| Uptime | Event | What it means |
|---|---|---|
| 17.926 s | +CGEV: ME PDN ACT 0,0 | Default PDN context (CID 0) activated |
| 18.428 s | +CGPADDR: 0,"10.x.x.x" | IP address obtained on CID 0 |
| 18.804 s | +CGEV: ME PDN ACT 1 | A second context (CID 1) appears, which nobody asked for |
| 18.809 s | Application | Network up: opening the DTLS session |
| 19.203 s | +CGEV: ME PDN DEACT 1 | CID 1 deactivated, right in the middle of the handshake |
| 19.303 s | +CNEC_ESM: 36,1 | ESM cause 36: “regular deactivation”, not an error |
| 21.948 s | Application | First data sent: failure |
Affected SIM card (SGP.02). With the reference card, the same sequence runs without any CID 1 event, and the session comes up normally.
A secondary PDN context — a second “data pipe” to the network — is opened and then closed about 400 ms later, without our firmware ever requesting it. No AT+CGDCONT command, no secondary context management on our side.
Who opens this context?
The notification says ME PDN ACT (Mobile Equipment) and not NW PDN ACT: it is the terminal itself that activates the context, not the network. Combined with a retry pattern typical of client-side logic (every ~30 seconds, then backing off), this points to the SIM card: M2M cards compatible with GSMA SGP.02 (remote SIM provisioning) can ask the modem, through a SIM Toolkit command (OPEN CHANNEL, BIP protocol), to open their own data context to reach their provisioning platform.
So this activity is legitimate, and it isn't ours. But it raises a much more interesting question.
The real question: why does a context we don't use break our connection?
Our session runs on the default context, CID 0. CID 1 belongs to the SIM card. Its deactivation should be a non-event for us — and the modem traces confirm that CID 0 stays active throughout. Yet our application behaves as if the network had disappeared.
So something between the modem and our application was misinterpreting this event. Time to go read the SDK code.
In the nRF Connect SDK, the lte_link_control library parses the modem's +CGEV notifications and turns them into LTE_LC_EVT_PDN events. These events are consumed in particular by lte_net_if, the layer that exposes the modem as a Zephyr network interface (net_if) and plugs it into the connection manager (conn_mgr).
And that's where the bug was: lte_net_if handled every PDN event as if it concerned the default context, without ever checking the context identifier (CID). When CID 1 was deactivated, it did exactly what it would do if the main connection were lost:
- removal of the interface's IPv4/IPv6 addresses;
- switch of the
net_ifto dormant state; - propagation of an L4 disconnect event (
NET_EVENT_L4_DISCONNECTED) to Zephyr and to the application.
Our application, doing exactly what it should when the network disappears, tore down its DTLS session. The bug wasn't in our code — but it took knowing our code, Zephyr's network stack and the SDK's modem layer to find it.
Why didn't anyone see it sooner? Most SIM cards never open a secondary context. The bug only shows up with SIM cards that do — SGP.02 M2M cards, but also any application that manages several PDN contexts itself. A classic case of a bug that hides in the combination of components, not in any single one of them.
The fix: six lines
Once the diagnosis is made, the fix is almost disappointingly simple. lte_net_if only manages the default context: it simply needs to ignore the others.
@@ -332,6 +332,12 @@ static void lte_evt_handler(const struct lte_lc_evt *const evt) break; case LTE_LC_EVT_PDN:+ /* We only handle default PDN (CID 0), ignore others */+ if (evt->pdn.cid != 0) {+ LOG_DBG("Filtered PDN CID %d event: %d", evt->pdn.cid, evt->pdn.type);+ break;+ }+ switch (evt->pdn.type) { case LTE_LC_EVT_PDN_ACTIVATED: LOG_DBG("PDN connection activated");
We validated the fix on our gateways with the four affected SIM cards: the application session no longer drops, and CID 0 stays up from start to finish. We could have stopped there, with a local patch in our west manifest. We chose to go further.
Upstreaming: from patch to pull request
Fixing a bug in your own fork takes an afternoon. Getting it accepted by the maintainers of an SDK used by thousands of products is another exercise: the fix has to be right for everyone, not just for us. Here's how it went.
-
Jul. 24, 2026
The PR is opened on nrfconnect/sdk-nrf
First version, more ambitious: a guard in
lte_net_if, and a second one inlte_link_controlso it only reports events for the contexts it manages itself. As an external contribution: CLA signed, and CI is only triggered after a Nordic maintainer explicitly allows it. - Same day First feedback: commit format The compliance check rejects our commit messages: lines over 72 characters. Trivial, but it's a good reminder that every project has its conventions, and they're not optional.
-
Jul. 27
The unit tests catch us out
The
tests/lib/lte_lc_pdntest suite fails. We'll be honest: we hadn't run Twister locally on that part. The test revealed that our change tolte_link_controlaltered its public behaviour for code that relied on it. The question goes up to Nordic's modem team. -
Jul. 28 → Aug. 6
Review: less is more
The reviewers' recommendation: keep only the change in
lte_net_if.lte_link_controlmust keep reporting every PDN event — other applications may legitimately need them. It's the consumer of the events that has to filter what concerns it. -
Aug. 6, 2026
Approved, merged, backported
One commit, one file, six lines. Three approvals from Nordic engineers, CI green, merge into
main. Since the change requires no adaptation from existing applications (no API, Kconfig or devicetree changes), it is backported to thev3.4-branchfor the next patch release.
The final version is smaller, and better, than the one we submitted. That's exactly what code review is for: our initial fix fixed our problem; the merged fix fixes the problem without creating one for anyone else.
Bonus: fixing the stack revealed the real impact
Once the session no longer dropped on every CID 1 event, we could watch the gateway over long captures — and see what the bug had been hiding until then.
The SIM card retries its secondary context about every 30 seconds, four times, before backing off. Each attempt wakes the radio up. With an active timer (T3324) of 3 minutes negotiated with the network, these wake-ups delay entry into PSM, the modem's deepest sleep mode, by several minutes on each cycle. For a battery- or solar-powered device, that's directly measurable on the energy budget.
That part isn't solvable from the terminal: it depends on the SIM card's provisioning profile. But thanks to the fix, we were able to send the SIM provider a precise, decoded and quantified report, rather than a vague “some cards don't work”.
Why we contribute upstream
Keeping a local patch would have been faster in the short term. Here's why we never settle for that:
- A local patch is technical debt. Every SDK update has to carry it forward, test it again, and hope the code around it hasn't moved.
- The bug isn't ours alone. SGP.02 SIM cards are commonplace in industrial IoT. Every nRF91 product using them was exposed to the same issue.
- Review improves the fix. Our final patch is better thanks to the people who know this code best.
- It's the proof that we master our stack. Zephyr, the nRF Connect SDK, the modem, the SIM card: when something breaks, we go down to the root cause, whichever layer it sits in.
“The libraries we build on are not black boxes. When we find a bug in them, we fix it — and we fix it for everyone.”
— Axel Moinet, PhD in Computer Science, CodiumUsing open source in industrial products comes with a responsibility: understanding it, and giving back to it. A six-line fix accepted upstream is worth more than a hundred lines of patches maintained in our corner.
At Codium, firmware, electronics and RF live under the same roof, in Langres. This is what lets us follow a bug from the antenna all the way to the application code. Developing a connected product on nRF91, nRF54 or Zephyr? Let's talk.