My SINCON 2025 Experience

I recently got to experience SINCON 2025 in Singapore!

Massive shout-out to @Sherpasec for giving me a ticket 🥳

Despite having been in the industry for a bit (I used to be a Researcher at a Cybersecurity company, but long story short, I’m a Software Engineer at an unrelated industry now), I’ve never actually been to a cybersecurity conference before this, which is something that always surprises my friends when I tell them. It was a bit intimidating; I’m not a very social person, and as I understand it, the main draw of conferences is not the excellent talks and workshops (which are usually uploaded online afterwards anyway), but the networking aspect. So to maximise my value of my ticket, I would need to try actually talk to people 😬😬😬

My friend (who is also my former colleague, before we both left the company for other opportunities), was doing the Malware Village at the event, and it was actually him who asked me if I wanted to go. I said yes, applied for leave from my job, and booked a flight and shared a hotel room with him!

I’ve not been to Singapore in a while, and usually when I do, I’m usually part of a group with a strict schedule, so it was fun to travel around Singapore. From the excellent public transport, meeting up with friends who live here, looking for good affordable food (Malaysia wins in this regard!), and just generally roaming around.

It was surreal to be able to walk around the event, and every corner there is something interesting going on! Across my friend’s already excellent Malware Village booth (with some fun reversing of some real malware! Something I used to occasionally do at my previous job.), there was a talk on Bug Bounty by YesWeHack, with some solid advice.

Across the hall, there was a hardware hacking village, where i spent some time attempting to solder some LEDs on one of their PCBs.

Emphasis on “attempting”! Apparently I short-circuited the PCB somehow?? In my defense, I haven’t soldered since Secondary School, over a decade ago. They thankfully gave me a free replacement, so I wasn’t out of 40 SGD :x

I also spent a bit of time doing their CTF, and solved one of their Reversing Challenges, before deciding it would be better use of my time to walk around and see the talks and workshops. I could always do CTFs at home!

Also, I was kinda discouraged at the time from not being able to solder the LEDs properly (Which was one of the challenges), and also that I was not able to clone the RFID due to not having an Android phone (They offered a Proxmark, but I was having trouble setting up the software on my machine). The sad thing is, back home, I have both a spare Android phone, and an M5StickC Plus2 with an RFID module (It’s an ESP32 chip in a neat package with some buttons and a display - think of it like a poor man’s Flipper Zero).

My coworker actually saw the USBs they were giving out for the Malware Village on Twitter (now known as X), and remarked on how cool it looked! He was surprised when I told him I was friends with (and came with, lol) the host at the booth who made and brought said USBs, and asked me to save him one if there were any extras at the end of the event.

I legit forgot to take photos of my friend’s booth or USB or anything and I feel so bad for that lol. So here’s a screenshot of the site (not sure how much longer it will be up). I remember having trouble with one of the flags, and the issue turned out to be that i was using forward-slashes for the directory (C:/Users/User/Example) instead of backward-slashes (C:\Users\User\Example), which is some weird Windows thing, that I, as an enlightened Arch user, would never understand :P

I remember I was listening to a speaker talk about North Korean APTs, when a crew member came up to me and asked if I remembered her. Cue me panicking and scrambling in my brain trying to piece together where I have seen this unfamiliar face before, hahaha, before eventually being told they were the partner of one of my friends.

I was also excited to go to a WinDBG workshop. I had learnt how to use WinDBG previously, when doing my OSED certification a few years ago (Side-Note: It was very fun! Totally recommend to anyone interested in Windows binary exploitation!), but with the older WinDBG UI. And I have played around with it (and the newer UI) for fun in the past few years (mostly because its one of the few debuggers able to debug the Windows kernel), and occasionally using it at my previous job. But I would not say it has ever been a core skill I learnt to use super competently.

I’ve been calling it Win-Debug, or less commonly, Win Dee-Bee-Gee, all these years, but the speaker called it “Wind-bag”. I looked it up online, and apparently old-timers (including Raymond Chen) call it that. If it’s good enough for Raymond Chen, its enough to convince me of its legitimacy as a pronounciation of WInDbg.

It was a nice refresher on how to use WinDbg, which I’ve mostly forgotten over the years. the speaker was really cool, an ex volunteer firefighter (This doesn’t exist where I come from, so I couldn’t help but think of the VFD from A Series of Unfortunate Events XD) and vulnerability researcher at CrowdStrike. They wrote a tool, DrawMeATree, using Graphviz and Python, that was nice visualisation in a graph of the wt command in WinDbg, which traces what calls are made, and its hierarchy of how deeply nested it is (The difference between a function graph in IDA/Ghidra/Binja, being only what is executed is traced and logged). The idea being that, you can filter out functions that you deem unimportant, and it will hide that function (along with every call inside that) from view, allowing you to focus on finding what’s important. The speaker shared an anecdote their mentor told them, of how a good Reverse Engineer could reverse an entire binary, but a great one would be able to reverse only exactly what is needed.

It reminds me a bit of the flamegraphs, which are a profiling tool for visualising stack traces, with the main difference being that flamegraphs are more for finding what functions are taking up a lot of time, so it is visualised a bit differently.

It does make me think of writing my own tool to do this. I initially thought to myself why it wasn’t using something like DynamoRIO/Frida, or a PyKd WinDbg plugin. At the next talk (more focused on the DrawMeATree tool), the speaker mentioned they wrote it in Python with Graphviz because of ease-of-use and installation, with just copy-pasting the output of the command into a file. Nonetheless, I think this would be an interesting exercise to try do using the frameworks I mentioned earlier.

I’ve been reversing a GBA game (Blog post pending!) and have been frustrated trying to dig through to find certain functionality I want to reverse, and I think something like this would be a big help. Though, I’d have to port the functionality of the wt command to gdb, or patch a GBA emulator to trace it.

I also wonder whether the use of trace-diffing might be a useful add-on here. Maybe you want to figure out how taskkill’s /FI argument (which applies a filter to processes to be terminated, e.g: specific window titles) works, so you run it once with that argument with some sort of filter, and once without it. Then have the tool find where the code path diverges (or what function calls they dont have in common).

In spirit of being “easy-to-use”, I think there is an even easier way to do it, rather than having to install python, the dependencies in requirements.txt, and graphviz, and having to re-run the command with different arguments every time you want to add a filter.

The web!

I can imagine a single static html page (and maybe a .js file), would be enough to have a nicer interface for this.

I personally despise web-development, and think modern web-development is super-bloated (That’s a topic for another day though). But one of the interns at the company I work at had previously positively raved about the Javascript data visualisation library d3.js (See: https://d3js.org/) when working on their feature of making an interactive histogram for my workplace’s product. Plus, it just looks really nice and interactive. Setup would be even easier too- just copy paste the wt output onto the website, and you will get an interactive graph, which you can click on each nodes and hide paths as needed, and still have the ability to add a list of strings (that you can add to with a textbox) that you can filter out functions that contain a specific string - and this time without re-running any commands on the terminal.

The downside is that it would be harder for automation. I imagine a scenario where you might want to automatically generate this function graph for several different versions of the same executable, on-demand, as they get updated. You can have the script run WinDbg and get the wt output, and pipe it into the DrawMeATree tool, and generate the graph automatically, and be able to see what differs across different versions. But I suppose doing this with d3.js might still be possible with node. And maybe this scenario is unrealistic, and it’s meant more as a one-off tool to quickly find a function that you’re interested in.

Speaking of work, there was a last-minute talk about fuzzing, because the original speaker (on a different topic) could not make it. I’ve played around with AFL++ before, and in my workplace, I code on a (very) large C++ codebase. I remember looking at our library dependencies, and noticing that several of them (such as nlohman’s json parsing library, harfbuzz, libxml2, libpng, among others.) have started incorporating fuzzing into their projects’ CI/CD pipeline.

For example, if you go to nlohmann’s json library github repository, and check out the pull requests page and look at a pull request with code changes, you would notice that alongside a check in the pipeline for Unit Tests, build success, linter checks, a static code analyzer, and other stuff, there is also job for fuzzing. If any of these checks fails, your pull request would be denied, and won’t be able to be merged back into the main branch.

I took the opportunity to ask the speaker for advice on this matter and whether it might be something we should look into doing. To my surprise, they did not seem too keen on the practice, citing the potential slowdown in development time. This is something I deeply empathise with, as, at work, it takes several minutes to compile even a few lines of change of code. Building from scratch takes a couple hours (even with our beefy setups!), and our pipeline runs for hours before we figure out if we broke any unit tests or e2e tests, at which point we start again. Adding a fuzzer to the mix would make this time even longer! Plus, the speaker mentioned it isn’t an exact science on how long it takes to run the fuzzer. Maybe you run it for 3 hours, but it would have been 10 hours before it found a bug you introduced! They also mentioned it being difficult to automatically detect what parts (sources/inputs) the developer added in, to be able to only fuzz that part. I suppose it could be done explicitly by the developer, similar to how they add unit tests for the feature they added, they could add code to say exactly what new parts to fuzz for, but this would be another point of friction for the developer to remember to do; we already (mostly) don’t like adding unit/integration tests!

When speaking of the topic to a friend later on, they suggested as a compromise, to have the fuzzing be done by a separate team or at separate points, instead of at every pull request. I imagine something like, before every stable release, we can start fuzzing at the commit we plan to use for the release, and fix any bugs as it finds them.

Side-Note: While explaining this and giving their thoughts on integrating fuzzing in the pipeline and discussing whether too much testing slowing down developer momentum is a good thing, they made a snide joke about CrowdStrike and their famous IT Outage incident. What they didn’t realise, was that there was a CrowdStrike employee right behind them who- thankfully, either didn’t hear what was said, or chose to ignore it.

Nonetheless, I think I would do more fuzzing exploration in the future. I remember reading that OpenSecurityTraining2 has started beta-testing an AFL fuzzing course (Fuzzing 1001. I didn’t sign up for it, because I was/am busy with work. I did sign up for some other OST2 courses as part of my work’s KPI tasks though, they have courses specifically targeted for C/C++ developers), and I believe Google provides a cash reward for incorporating a fuzzing harness into critical open source projects too, through OSS-Fuzz.

I have my eye on a popular-in-a-niche-industry library we use at work and I have been working closely with over the past few months, that deals with a very niche file format and binary network protocol. Already, when learning how to use the library (not even actively looking for vulnerabilities!), I have found at least one legitimate potentially exploitable bug. The reason I had to look at the source code is because documentation and example code online is sparse. The library is really old and makes…interesting design decisions that are prone to mistakes. Though I suppose I can’t really blame them, since the library predates modern C++ and widespread good practices being more commonplace. I have also seen security researchers find exploitable bugs in the library in recent years, possibly also with fuzzing, based on the nature of the bugs, so I do think there are more potential vulnerabilities hidden in there.

Anyway, that was the conference. There were several things I omitted, like my experience analysing the malware, or the Android Reversing workshop I attended half-of before unfortunately deciding to head elsewhere (It was still good! I just had some place else I had to suddenly be, sadly), and other things. But unfortunately, I could not fit everything into my blog post - it is already long enough as it is.

Also I want to mention this really nice Scrambled Egg Rice Keisuke I went to afterwards. Yes, I decided this is worth mentioning.

And also, my friend told me I should check out Jewel at Changi Airport since there was time before the flight. It’s really nice, no regrets taking a detour to come here.

Oh, also I want to mention that there was a Nintendo thing at the airport at Jewel, where there were “stamps” at every floor, and you could collect them to get a prize.

Sadly, they ran out of stock of the prizes (Nintendo Stickers!) - I believe we were the first ones not to get it. But at least we got to keep this cool shiny Nintendo stamp thing!

Anyways, directly after SINCON, I took a flight “home” to Malaysia (in the state I live/work in, Kuala Lumpur), rushed to buy a laptop charger because I lost mine having forgotten and left it in the hotel, got home at night, and had to wake up at 5am the next morning to rush and pack yet again, for a trip back to Sabah for Harvest Festival (Kaamatan), which was where I was born and raised- mostly. I’ll leave you with a picture of the sunset from my hometown :-) Thanks for reading!