Earlier this year, HackTheBox (HTB) announced its latest certification - its first in examining the intersection of artificial intelligence (AI) and security. The Certified Offensive AI Expert (COAE) credential made for a natural milestone in my ongoing efforts to cross examine AI in the cybersecurity space.
If you think securing an AI model starts and ends with the prompt, you are already behind 🏃♀️
— Hack The Box (@hackthebox_eu) April 27, 2026
Attackers are finding ways to compromise the system from the initial training data to the final output. We are breaking down the major AI attacks you cannot afford to ignore 👇
Level… pic.twitter.com/9EVAfJFNqY
Given my previous experiences with HackTheBox’s training offerings with the CPTS and CBBH (now CWES), I had my expectations set pretty high for this exam’s content.
Having passed the exam, I thought I’d catalog my thoughts on Hack The Box’s latest offering below.
COAE Training Material: HTB Academy#
Arguably, the greatest value for pursuing this certification isn’t the credential itself, but the accompanying compulsory training modules hosted on HackTheBox’s Academy platform. As of the time of writing this, the COAE’s training curricula is coupled with the content found within the Academy’s AI Red Teamer path, which - as of writing this - incudes such topics as:
- Fundamentals of AI
- Applications of AI in InfoSec
- Introduction to Red Teaming AI
- Prompt Injection Attacks
- LLM Output Attacks
- AI Data Attacks
- Attacking AI - Application and System
- AI Evasion - Foundations
- AI Evasion - First-Order Attacks
- AI Evasion - Sparsity Attacks
- AI Privacy
- AI Defense
HTB Academy is a really stable and feature-rich training platform. If you’re not otherwise familiar with it, within its various “modules” HTB offers guided training to topics, typically subdivided into pertinent sections and paginated by topic. For example, the Fundamentals of AI module splits-up its training into the sections of Supervised, Unsupervised, and Reinforcement Learning Algorithms (among several others); within the Supervised Learning Algorithm section you’re meant to drill down deeper into pertinent topics like Linear Regression, Logistic Regression, Decision Trees and Naive Bayes: topics which classically make up the foundations for the Supervised Machine Learning. Many sections will have a hands-on practical application of what was learned, with an overarching skills assessment for the entire module marking the close (usually creatively applying what was covered in the guided sections).
Content Covered#
At 12 modules, the amount of ground to cover is quite a bit less than some of HackTheBox’s other certifications (22 for the CWES, 28 for the CPTS); however, I would argue that some of the content covered is substantially more dense than what you might encounter in some of the Academy’s other offerings. HackTheBox’s curricula is quite ambitious in seeking to condense multiple semesters-worth of Computer Science education into its training. While some modules - like the Prompt Injection Attacks - are fairly easy to understand and execute, there are several which include some pretty advanced mathematics (or at least, high-level mathematical notation than what most who explore the space of cybersecurity bother with learning). For example, consider the following formula HTB presents on learning about AI Evasion (and more narrowly, L1 Proximal Operators):
Fortunately, in most cases HTB splits apart what’s necessary for understanding the underlying concepts from engineering such methods in code; most of the time, HTB has done the courtesy of already drafting much of what you need to do for you (if only requiring you to stitch-together the disparate code blocks they pre-define for you). While I’d encourage anyone who is genuinely interested in security research in AI to grapple with the math, it’s not explicitly necessary to get through the modules and pass the certification. Ironically, it would not surprise me to learn of students cognitively offloading the lessons imparted to an LLM to handle.
I think the best lessons HTB has to offer within the AI Red Teamer Academy path are those that deal with attacking the model directly (either in their training corpus data or in creating shadow adversarial models), as these explore more into the domain of AI Security research advancements. However, a valid criticism that has emerged from HTB’s customer base has been how applicable these topics are to real-world test events; how often - for example - is a Pentester going to be able to perform a Jacobian-based Saliency Map Attack (which requires white-box transparency of the underlying model)? There are also some instances where I would question just how fundamental some of the explored research really is; some of the topics covered in the more complex modules felt like they were cherry-picked as an interesting research edge-case vs. something representative of security research at-large. A lot of time is exhaustively spent looking at various ways of attacking image classifier models, but none is spent on vulnerabilities present in models used for video and audio media; I was surprised that there was nothing concerning how to perform deepfakes of someone’s voice for phishing (or how to understand/bypass facial recognition models), for example. I feel that there is a lot of room to grow for the associated training path in these areas.
A lot of time is exhaustively spent looking at various ways of attacking image classifier models, but none is spent on vulnerabilities present in models used for video and audio media
By contrast, exploiting model behavior by way of prompt injection wasn’t as interesting or challenging (but are more in-line with what classic pentesting might aim to accomplish); there’s plenty of content there for those who haven’t explored these topics before, but I’d argue there are vendors who present a more diverse range of ways to attack such models (and under more hardened conditions). For those interested in those topics, I’d assert better opportunities to explore those kinds of attacks are available through Gray Swan and Lakera.
One thing you shouldn’t expect to have integrated into the training (and is largely treated as as known knowledge by the student) are some of the more traditional security vulnerabilities that AI models might exacerbate. Students enrolling into the AI Red Teamer course are just presumed to know/understand command injection, path traversal, and cross-site scripting, for example. If you don’t know how to do these things, you probably should consider walking back from this training offering until later.
COAE: The Exam#
HTB’s terms of service explicitly prohibit me from disclosing details about the exam that they don’t otherwise share themselves; as such, there’s not much more I can say that others haven’t. If you’ve taken any of HTB’s other exams, their format should be quite familiar to you: you’re put into a scenario with a fictional customer who - through a detailed Letter of Engagement - scopes the work you’re meant to perform. Benchmarking that work are a series of tasks (which are met through the attainment of CTF-like flags), which help point you in the direction your work is meant to take you. Each flag is worth a certain number of points, with the minimum passing threshold being 85 out of 100 points. In addition to simply scoring points, you are required to submit an after-action report (not unlike what you might for a real-world test engagement) detailing your various findings and exploit chains. Each exam voucher you purchase entitles you to two attempts at passing the exam (this is standard across all of Hack The Box’s certification exams); even if you fail on your first attempt, Hack The Box’s reviewing staff provide feedback on where/how you could improve your performance based on the content of your report.
The exam’s testing environment is available to you for up to 7 calendar days, which allows you to comfortably arrange your applied time testing around other competing things that might be happening in your real life. Unlike some of its other exams, HTB doesn’t make use of a VPN for you to engage the test environment (simply configuring your /etc/hosts file to point at the appropriate domains and subdomains will suffice).
Exam Experiences#
For as much as I liked HTB’s Academy platform for its quality of training materials and exercises, I feel conflicted about the COAE exam and certification.
Is it hard?#
I felt that this exam was much easier than the exams put forward for the CPTS and CWES exams, respectively. Whereas it took me multiple attempts to pass the CPTS and CWES exams, I finished the COAE exam well within the alloted time window and submitted it days ahead of the deadline. Overall, I’d argue my experiences with HTB’s weekly machine releases to be of greater challenge than the exam environment provided.
One of the reasons I feel that the exam is so much easier than the others is owing to its narrow scope of testable learning objectives. For the CPTS/CWES, if you get stuck you’re often pouring over massive indexes of notes trying to discern what you may have overlooked or failed to consider; at 28 and 22 modules worth of content respectively, there’s a lot of potential avenues to explore and possible vulnerabilities to check and iterate upon. At only 12 modules worth of content in the aligned AI Red Team path, there’s just less to have to consider for the COAE exam (with some of it blatantly apparent as to what you’re meant - or not meant - to consider).
I also felt that the overall scenario architecture was more convoluted with the CPTS/CWES exams; the attack surfaces for those exams had a lot of sprawl to them. This helped add layers of uncertainty to those exams, which both elevated their difficulty and contributed to their realism; you could never be sure if you were stuck because you had incorrectly implemented an exploit, because you had overlooked a class of vulnerability covered from the instructional materials, or because you hadn’t yet uncovered a crucial piece of information hidden elsewhere in the test environment. With little exception, this wasn’t my experience with the COAE exam.
Environment Instability#
Julian Gomez mentioned this in his own review of the exam, but I had to actively battle the test environment itself while conducting my exam. BurpSuite - for whatever reason - regularly drops requests in its proxied browser (rendering its classic white-and-orange site with a “No response from server” message coming back). While refreshing the page was often remedy enough, it made for quite a frustrating experience in enumerating content served over the web.
Another issue I encountered was that my test environment was actually broken at one point; without going into too much detail, a key backend service had become downed and I was unaware of whether that was deliberately the case or not. I lost a whole day enumerating the exam environment trying to figure out what had happened - and even got assurances from HTB’s own support staff that nothing was wrong - before resetting my exam environment and resolving the issue.
Arguably, this was user-error on my part; this wasn’t the first exam I’d come across that has had issues fixed by an environment reset (hence why I had suspicions that resetting might work to begin with). However, it’s worth calling out just in case you might likewise encounter such trouble. I have little doubt that - in time - HTB will be able to iron-out these issues.
Report Writing#
Do not shirk writing the report. As embarassing as it is to admit, I actually failed my first attempt at the exam not because I didn’t have enough points to pass (I did), but because apparently the quality of my report was lacking. Let me serve as a cautionary warning that Hack The Box is willing to fail someone who obtains a passing score but doesn’t have a report that rises to their standard of quality. I was definitely annoyed at that. If that had been my final attempt of the two alloted per exam voucher, I probably wouldn’t have bothered purchasing another voucher to re-attempt the exam again.
Let me serve as a cautionary warning that Hack The Box is willing to fail someone who obtains a passing score but doesn’t have a report that rises to their standard of quality.
While I wish I could share what my report had looked like for others to benchmark against, I can’t - as that would go against the vendor’s Terms of Service. However, I can generalize for you all:
- Using Hack The Box’s provided template isn’t explicitly necessary, but it’s probably advisable. The provided template is a guideline for how to frame your final report; you do not need to include every section/subsection. My rejected report included almost all of the out-of-the-box formatting; my accepted report was more tailored and omitted sections I felt didn’t add anything.
- Be explicit and verbose in detailing your findings. I have developed a habit over time in rolling smaller findings that relate/chain-together with one another into a single, monolithic finding (this is a courtesy for developers so they can fix multiple related issues in a single ticket vs. multiple tickets). This approach did not serve me well on my first report. Breaking down (and cross referencing) the individual findings that lead to a “high” risk finding proved to be more acceptable (and - consequentially - more than doubled the number of findings I had initially reported); presenting the final finding as utilizing Finding X, Finding Y, and Finding Z is more desirable.
- Your findings should construct a narrative - a writeup of sorts which allows the reader to replicate your findings from scratch. Professionally, I don’t typically include such summaries of how I discovered a vulnerability vs. how I exploit the presence of a vulnerability; that’s more the domain of CTF writeups for developing a testing methodology than actually fixing an exploitable risk. However, this approach is not sufficient enough for the exam reviewers, which I understand. They already know it’s vulnerable, the narrative isn’t to inform them of something that’s fixed but to assure them that I haven’t cheated - that I actually know how to achieve the score that I did.
- Though they don’t explicitly say this in the instructions, you shouldn’t include any large scripts/code in the main body of the report. My accepted report included such code as an Appendix where they were referenced in the report body as needed (i.e. “see Appendix B”).
Is it worth it?#
This is an issue that I think all of HTB’s certifications struggle with justifying. Outside of upskilling, the value of a certification for most end consumers will be in how well its attainment will promote their own individual employability. It’s no secret that there is no shortage of vendors and certification offerings available for people to spend their time/money on acquiring, but only a narrow subset of them actually matter to individual employers.
Hack The Box is still relatively new to the space - its first available certification (the CBBH, now CWES) was released in 2022 and as of the time of writing this post only has been passed by about 2k people (it’s most-issued certification, the CPTS also released in 2022 is doing only slightly better, at just shy of 3k people); but their exclusivity - which some market as a sign of its higher-caliber/standards - hurt their marketability for cert-holders, because then its not a quantitatively strong enough signal for employers to filter against. For comparison’s sake, OffSec said in 2025 that “tens of thousands of people have achievd the OSCP certification”; CompTIA championed how more than a million people attained their Security+ cert in 2025; in 2024, ISC2 announced their CISSP certification had been attained by more than 165,000 people worldwide. I’m aware that I’m making apples-to-oranges comparisons in some of these cases (e.g. the target demographic for the Security+ is far less technical, less offensively-oriented, and more early in their career than what HTB’s certifications generally target), but the point is that there is significantly broader appeal for these other certs and vendors than what HTB can offer with theirs.
Finally, I’ll hazard that professionalized security research in the AI space is largely not driven by certifications (vs. paper-publishing, conference presentations, etc.). Ultimately, I’m not really convinced that attaining the COAE certification will significantly improve one’s employability at getting into AI Security. I think the real winner of attaining one of HTB’s certs is HackTheBox itself: the certs serve as a draw to their Academy platform (which - again - I think is incredibly well-curated, very accessible, and has a wonderful community backing it) and to engage/explore content on the platform you might not otherwise consider.
Closing Thoughts#
This year has really kicked-off my exploring all manner of AI-related security studies and I was quite pleased with the materials provided by Hack The Box in support of the COAE certification. The content to learn is surprisingly involved and has quite a bit of depth to it; the real value to the COAE is in that training. But where HTB seeks to explore how security concerns can emerge in a model’s conception and development, there is some struggle with the COAE exam implements those in practice. Overall, while I’d encourage people interested in the intersectionality of AI and cybersecurity to review the Academy content, you can probably save your money on a voucher for the exam.
