The modern security operations center is under relentless pressure to ingest, analyze, and respond to an ever‑growing tide of alerts. Traditional workflows that rely on human analysts to examine each suspicious file, match it against internal policies, and decide on a remediation path have become untenable as volume and sophistication of threats increase. Recognizing this bottleneck, many enterprises are turning to automation frameworks that can handle routine checks and enforcement actions, freeing skilled personnel to focus on the most strategic investigations. In this evolving landscape, the integration of large language models into the triage pipeline represents a natural progression: these models bring a breadth of world knowledge and an ability to follow complex instructions that can be harnessed to make sense of ambiguous software identifiers. The case of SoftBank Corp. illustrates how a purpose‑built, on‑premises LLM can be woven into an existing automation fabric to achieve end‑to‑end handling of low‑level triage tasks, thereby reshaping the economics of security operations and setting a benchmark for peers seeking to balance speed, accuracy, and compliance.
Before the introduction of any automated assistance, SoftBank’s analysts were tasked with a triad of repetitive activities for every flagged executable or script: extracting the file name, determining its functional category, verifying whether that category corresponded to an allowed or prohibited behavior under corporate policy, and then initiating the appropriate containment or remediation step. Each of these stages required careful attention to detail, cross‑referencing of internal asset inventories, and often a manual lookup in threat intelligence feeds. The cumulative effect was a significant drain on analyst bandwidth, with senior staff spending countless hours on low‑value classification work that could have been delegated to a machine. Moreover, the monotony of the task increased the risk of oversight, especially during shift changes or when alert volumes spiked. By isolating the categorization step as the final obstacle to full automation, SoftBank identified a clear opportunity to apply artificial intelligence where human effort was both costly and error‑prone, paving the way for a more resilient and responsive security posture.
Large language models have emerged as versatile tools for a variety of natural language tasks, ranging from summarization to code generation, and their underlying architecture makes them particularly suited for classification problems that involve nuanced semantic understanding. Unlike rule‑based engines that require exhaustive enumeration of every possible software variant, LLMs generalize from the patterns they have seen during training, allowing them to recognize unfamiliar program names by inferring similarity to known categories. This capability is invaluable in a security context where attackers frequently rebrand or slightly modify malicious tools to evade signature‑based detection. Furthermore, LLMs can be guided through explicit instructions and examples, enabling organizations to steer the model’s output toward a predefined taxonomy without the need for costly retraining. For SoftBank, the appeal lay in harnessing this general‑purpose reasoning power while keeping the model’s behavior tightly controlled through prompt engineering and validation loops.
Data sovereignty and privacy concerns often dictate that sensitive security telemetry cannot leave the corporate perimeter, especially in jurisdictions with strict data protection regulations or in industries where intellectual property is paramount. Cloud‑hosted LLM services, while convenient, introduce uncertainties about data residency, potential logging by third‑party providers, and the risk of inadvertent exposure through model APIs. Consequently, SoftBank required a solution that could be installed and operated entirely within its own data centers, ensuring that every inference remained under its direct governance. An on‑premises deployment also offers predictable latency, easier integration with existing security orchestration tools, and the ability to audit the model’s behavior for compliance purposes. By selecting an open‑source model that could be packaged, hardened, and run behind firewalls, SoftBank satisfied its internal risk‑management criteria while still gaining access to the cutting‑edge capabilities of modern language technology.
The model ultimately chosen for this initiative, Foundation‑sec‑1.1‑8B‑Instruct, distinguishes itself from generic alternatives through a combination of size, specialization, and practical engineering considerations. With eight billion parameters, it strikes a balance between representational power and computational efficiency, making it feasible to run on modest GPU infrastructure without incurring prohibitive energy or cooling costs. More importantly, the model has undergone additional pre‑training on a corpus rich in security‑related text—including vulnerability reports, malware descriptions, and security policy documents—which imbues it with a domain‑specific intuition that pure general‑purpose LLMs lack. This focused background enables it to discern subtle distinctions between, say, a legitimate file‑transfer utility and a tool that could be employed for data exfiltration, thereby delivering higher accuracy on the classification task at hand while still being lightweight enough for routine, high‑volume inference.
One of the core difficulties in automating software categorization stems from the inherent overlap between functional labels. Many applications possess multiple legitimate uses; a program designed for cloud storage might also be employed for peer‑to‑peer sharing, while a network‑monitoring utility could double as a packet capture tool for troubleshooting or malicious reconnaissance. When the taxonomy contains categories such as ‘File Transfer’, ‘File Sharing’, and ‘Forbidden Internet Service’, the model must decide which label best captures the predominant intent behind a given executable, a decision that often hinges on subtle contextual cues that are not present in the file name alone. Additionally, SoftBank maintains organization‑specific rules that further refine these boundaries—for instance, treating any cloud‑storage client as prohibited regardless of its sharing capabilities—adding another layer of complexity that a naïve LLM would struggle to respect without explicit guidance.
To coax the model into producing stable, correctly formatted outputs, the engineering team employed a prompt‑tuning strategy anchored in few‑shot learning. By appending a handful of carefully chosen examples—each pairing a software name with its correct category—to the end of the input prompt, the model receives a concrete illustration of the expected format and the semantic boundaries between classes. These examples act as a shortcut, reducing the reliance on the model’s internal priors and steering its generation toward the desired token sequence. Complementing the few‑shot demonstrations, a system‑level instruction block was added that explicitly states the output must be a single category name, with no extra punctuation or explanatory text. A validation wrapper then checks the model’s response; if the returned string does not match one of the seventeen permitted labels, the inference loop repeats automatically until a valid category emerges. This closed‑loop mechanism dramatically reduces the likelihood of malformed outputs and ensures downstream processes receive clean, predictable data.
Beyond formatting, the team enriched the prompt with explicit decision logic derived from veteran analysts’ heuristics and historical ticket data. For each ambiguous pair of categories, a series of conditional questions was encoded directly into the prompt, effectively turning the LLM into a lightweight rule engine that can be overridden by its learned knowledge when the signals are clear. For example, to distinguish between packet‑capture utilities and vulnerability scanners, the prompt first asks whether the software generates CVE‑style reports; if the answer is affirmative, the model leans toward the scanning label, otherwise it proceeds to examine whether the primary function involves intercepting or visualizing network traffic. Similar cascades were created for the file‑transfer versus file‑sharing dilemma and for differentiating legitimate server services from prohibited internet tools. By keeping these chains concise yet comprehensive, the prompt avoids overwhelming the model while still providing the granularity needed to cut through overlapping definitions.
The taxonomy includes a catch‑all bucket labeled ‘Undetermined’, intended to absorb any software that does not cleanly fit into the sixteen defined categories. In early experiments, the model displayed a reluctance to employ this safety net, often forcing an ambiguous item into one of the existing classes even when the evidence was weak. This tendency manifested as false positives in production, because the ‘Undetermined’ label carries no automatic policy action and thus a mis‑classification could lead to either an unnecessary block or an unwarranted allowance. Recognizing that the model’s drive to minimize uncertainty sometimes overrides the intention to leave a case undecided, the team introduced additional calibration steps. They adjusted the temperature parameter to encourage more conservative predictions and supplemented the prompt with explicit instructions that, when confidence falls below a dynamically calculated threshold, the model should default to ‘Undetermined’. This refinement helped align the model’s behavior with the operational need to err on the side of caution rather than over‑commit.
Even with prompt engineering and logical guards, certain organization‑specific edge cases persisted, particularly where internal whitelists or blacklists diverged from generic expectations. To address these residual errors, SoftBank instituted a two‑stage filtering pipeline that runs before and after the LLM inference. In the preprocessing phase, incoming file names are checked against a curated whitelist of known legitimate applications; any match is immediately classified according to the pre‑approved category, bypassing the model entirely and guaranteeing zero‑false‑positive handling for trusted software. Conversely, a blacklist of known malicious signatures triggers an automatic high‑severity response. After the model produces its category, a post‑processing step re‑examines the result against contextual attributes such as file provenance, signature scores, and recent threat‑intel feeds; if discrepancies are detected, the outcome is either overridden or flagged for secondary analyst review. This layered approach combines the breadth of the LLM’s generalization with the precision of deterministic rules, yielding a robust classification front‑end.
Empirical validation revealed that the Foundation‑sec‑1.1‑8B‑Instruct model, when operating in isolation, achieved an classification accuracy of roughly eighty percent on a held‑out set of historically labeled software names—a figure that rivals the performance of much larger, cloud‑based LLMs on the same benchmark. When the model’s output was fused with the rule‑based preprocessing and post‑processing layers described above, the overall workflow accuracy climbed to approximately ninety percent, meaning that nine out of ten alerts traversed the triage pipeline without requiring manual correction. This uplift translated directly into operational gains: analysts reported a noticeable reduction in the time spent on routine categorization, allowing them to allocate more cycles to threat hunting, incident response, and strategic security projects. The improved fidelity also decreased the frequency of false alarms, thereby lowering alert fatigue and enhancing confidence in the automated responses generated by downstream orchestration tools.
Looking ahead, SoftBank intends to extend this successful pattern to other facets of its security operations, most notably the automation of intrusion detection system (IDS) alert triage. IDS events often contain rich payloads—packet headers, payload snippets, and contextual metadata—that would benefit from the same blend of language understanding and rule‑based validation employed for file categorization. Because such data frequently traverses trust boundaries and may include proprietary network topology details, the on‑premises, privacy‑preserving nature of the Foundation AI model remains a decisive advantage. For other organizations contemplating a similar journey, the key takeaways are: start by identifying the most repetitive, rule‑heavy step in your existing workflow; evaluate whether an open‑source LLM can be hosted locally to meet compliance requirements; invest in prompt engineering that incorporates both few‑shot examples and domain‑specific decision trees; and surround the model with deterministic pre‑ and post‑processing filters to catch edge cases. By following this blueprint, security teams can achieve meaningful automation without surrendering control over sensitive data, ultimately shifting analyst effort from tedious classification to high‑impact threat mitigation.