Custom Cryptographic Protocol

From enterprise
Jump to: navigation, search
Custom Cryptographic Protocol
ID T1024
Tactic Command and Control
Platform Linux, macOS, Windows
Data Sources Packet capture, Netflow/Enclave netflow, Process use of network, Malware reverse engineering, Process monitoring
Requires Network Yes

Adversaries may use a custom cryptographic protocol or algorithm to hide command and control traffic. A simple scheme, such as XOR-ing the plaintext with a fixed key, will produce a very weak ciphertext.

Custom encryption schemes may vary in sophistication. Analysis and reverse engineering of malware samples may be enough to discover the algorithm and encryption key used.

Some adversaries may also attempt to implement their own version of a well-known cryptographic algorithm instead of using a known implementation library, which may lead to unintentional errors.1


  • has used a tool called RarStar that encodes data with a custom XOR algorithm when posting it to a C2 server.2
  • Several Lazarus Group malware families encrypt C2 traffic using custom code that uses XOR with an ADD operation and XOR with a SUB operation.3
  • 3PARA RAT will use an 8-byte XOR key derived from the string HYF54&%9&jkMCXuiS instead if the DES decoding fails.4
  • 4H RAT obfuscates C2 communication using a 1-byte XOR with the key 0xBE.4
  • BADNEWS encrypts C2 data with a ROR by 3 and an XOR by 0x23.5
  • BBSRAT uses a custom encryption algorithm on data sent back to the C2 server over HTTP.6
  • CORESHELL C2 messages are encrypted with custom stream ciphers using six-byte or eight-byte keys.7
  • ChChes can encrypt C2 data with a custom technique using MD5, base64-encoding, and RC4.89
  • CosmicDuke contains a custom version of the RC4 algorithm that includes a programming error.1
  • Derusbi obfuscates C2 traffic with variable 4-byte XOR keys.10
  • The C2 server response to a beacon sent by a variant of Emissary contains a 36-character GUID value that is used as an encryption key for subsequent network communications. Some variants of Emissary use various XOR operations to encrypt C2 data.11
  • uses fake Transport Layer Security (TLS) to communicate with its C2 server, encoding data with RC4 encryption.12
  • The original variant of FakeM encrypts C2 traffic using a custom encryption cipher that uses an XOR key of “YHCRA” and bit rotation between each XOR operation. FakeM has also included HTML code in C2 traffic in an apparent attempt to evade detection. Additionally, some variants of FakeM use modified SSL code for communications back to C2 servers, making SSL decryption ineffective.13
  • Some samples use a custom encryption method for C2 traffic using AES, base64 encoding, and multiple keys.14
  • Gazer uses custom encryption for C2 using 3DES and RSA.1516
  • Before being appended to image files, HAMMERTOSS commands are encrypted with a key composed of both a hard-coded value and a string contained on that day's tweet. To decrypt the commands, an investigator would need access to the intended malware sample, the day's tweet, and the image file containing the command.17
  • Hikit performs XOR encryption.18
  • The Komplex C2 channel uses an 11-byte XOR algorithm to hide data 19.
  • Lurid performs XOR encryption.20
  • RTM encrypts C2 traffic with a custom RC4 variant.21
  • Sakula encodes C2 traffic with single-byte XOR keys.22
  • Sys10 uses an XOR 0x1 loop to encrypt its C2 domain.23
  • Taidoor is known to utilize encryption within network protocols.24
  • httpclient encrypts C2 content with XOR using a single byte, 0x12.4


Network intrusion detection and prevention systems that use network signatures to identify traffic for specific adversary malware can be used to mitigate activity at the network level. Since the custom protocol used may not adhere to typical protocol standards, there may be opportunities to signature the traffic on a network level for detection. Signatures are often for unique indicators within protocols and may be based on the specific protocol used by a particular adversary or tool, and will likely be different across various malware families and versions. Adversaries will likely change tool C2 signatures over time or construct protocols in such a way as to avoid detection by common defensive tools.25


If malware uses custom encryption with symmetric keys, it may be possible to obtain the algorithm and key from samples and use them to decode network traffic to detect malware communications signatures.26

In general, analyze network data for uncommon data flows (e.g., a client sending significantly more data than it receives from a server). Processes utilizing the network that do not normally have network communication or have never been seen before are suspicious. Analyze packet contents to detect when communications do not follow the expected protocol behavior for the port that is being used.25


  1. a b  F-Secure Labs. (2014, July). COSMICDUKE Cosmu with a twist of MiniDuke. Retrieved July 3, 2014.
  2. ^  Counter Threat Unit Research Team. (2017, October 12). BRONZE BUTLER Targets Japanese Enterprises. Retrieved January 4, 2018.
  3. ^  Novetta Threat Research Group. (2016, February 24). Operation Blockbuster: Destructive Malware Report. Retrieved March 2, 2016.
  4. a b c  Crowdstrike Global Intelligence Team. (2014, June 9). CrowdStrike Intelligence Report: Putter Panda. Retrieved January 22, 2016.
  5. ^  Settle, A., et al. (2016, August 8). MONSOON - Analysis Of An APT Campaign. Retrieved September 22, 2016.
  6. ^  Lee, B. Grunzweig, J. (2015, December 22). BBSRAT Attacks Targeting Russian Organizations Linked to Roaming Tiger. Retrieved August 19, 2016.
  7. ^  FireEye. (2015). APT28: A WINDOW INTO RUSSIA’S CYBER ESPIONAGE OPERATIONS?. Retrieved August 19, 2015.
  8. ^  Miller-Osborn, J. and Grunzweig, J.. (2017, February 16). menuPass Returns with New Malware and New Attacks Against Japanese Academics and Organizations. Retrieved March 1, 2017.
  9. ^  Nakamura, Y.. (2017, February 17). ChChes - Malware that Communicates with C&C Servers Using Cookie Headers. Retrieved March 1, 2017.
  10. ^  Fidelis Cybersecurity. (2016, February 29). The Turbo Campaign, Featuring Derusbi for 64-bit Linux. Retrieved March 2, 2016.
  11. ^  Falcone, R. and Miller-Osborn, J.. (2015, December 18). Attack on French Diplomat Linked to Operation Lotus Blossom. Retrieved February 15, 2016.
  12. ^  US-CERT. (2017, November 22). Alert (TA17-318A): HIDDEN COBRA – North Korean Remote Administration Tool: FALLCHILL. Retrieved December 7, 2017.
  13. ^  Falcone, R. and Miller-Osborn, J.. (2016, January 24). Scarlet Mimic: Years-Long Espionage Campaign Targets Minority Activists. Retrieved February 10, 2016.
  1. ^  Somerville, L. and Toro, A. (2017, March 30). Playing Cat & Mouse: Introducing the Felismus Malware. Retrieved November 16, 2017.
  2. ^  ESET. (2017, August). Gazing at Gazer: Turla’s new second stage backdoor. Retrieved September 14, 2017.
  3. ^  Kaspersky Lab's Global Research & Analysis Team. (2017, August 30). Introducing WhiteBear. Retrieved September 21, 2017.
  4. ^  FireEye Labs. (2015, July). HAMMERTOSS: Stealthy Tactics Define a Russian Cyber Threat Group. Retrieved September 17, 2015.
  5. ^  Novetta. (n.d.). Operation SMN: Axiom Threat Actor Group Report. Retrieved November 12, 2014.
  6. ^  Dani Creus, Tyler Halfpop, Robert Falcone. (2016, September 26). Sofacy's 'Komplex' OS X Trojan. Retrieved July 8, 2017.
  7. ^  Villeneuve, N., Sancho, D. (2011). THE “LURID” DOWNLOADER. Retrieved November 12, 2014.
  8. ^  Faou, M. and Boutin, J.. (2017, February). Read The Manual: A Guide to the RTM Banking Trojan. Retrieved March 9, 2017.
  9. ^  Dell SecureWorks Counter Threat Unit Threat Intelligence. (2015, July 30). Sakula Malware Family. Retrieved January 26, 2016.
  10. ^  Baumgartner, K., Golovkin, M.. (2015, May). The MsnMM Campaigns: The Earliest Naikon APT Campaigns. Retrieved December 17, 2015.
  11. ^  Trend Micro. (2012). The Taidoor Campaign. Retrieved November 12, 2014.
  12. a b  Gardiner, J., Cova, M., Nagaraja, S. (2014, February). Command & Control Understanding, Denying and Detecting. Retrieved April 20, 2016.
  13. ^  Fidelis Cybersecurity. (2015, August 4). Looking at the Sky for a DarkComet. Retrieved April 5, 2016.