Tutorial¶

This tutorial helps you get started using MAGIC through a walk through of several capabilities MAGIC provides.

If you haven’t yet, first go through Accessing API.

Upload file for analysis¶

To start, let’s upload our binary to MAGIC. In addition to individual binaries, archives, such as zip or tar, are also supported.

curl

curl -F "filedata=@./sample.exe" \
     -F "origFilePath=sample.exe" \
     https://api.magic.cythereal.com/upload/${VIRUSBATTLE_KEY}

vbclient

vbclient -a upload ./sample.exe

This vbclient command will create a file in the local directory named UploadedHashes.txt that lists the SHA1s of all uploaded files. Additional uploads in the same directory will append to, rather than overwrite, this list. In addition, actions such as -a query or -a status will read hashes from this file if no arguments are provided. For the purposes of this tutorial, however, we will not make use of this feature.

Check status of Analysis¶

Further Details: Querying Analysis Status

In order to check the status of our uploaded file, we need to know the sha1sum of this file. On linux, this can be found using the sha1sum command. If the file is uploaded using vbclient, the UploadedHashes.txt file will contain this hash.

The hash for our example sample is 2d9035b27ce5c1cce2fb8432c76e315a1a2a8de0.

curl

To get the result of analysis:

curl https://api.magic.cythereal.com/query/${VIRUSBATTLE_KEY}/2d9035b27ce5c1cce2fb8432c76e315a1a2a8de0

This results in the following (See API Response Format for description of communication protocol.):

{
      "status": "success",
      "hash": "2d9035b27ce5c1cce2fb8432c76e315a1a2a8de0",
      "vb_version": "VB-0.4",
      "answer": {
            "sha1": "2d9035b27ce5c1cce2fb8432c76e315a1a2a8de0",
            "unix_filetype": "PE32 executable (GUI) Intel 80386, for MS Windows",
            "filepath": [
                  "./archive"
            ],
            "length": 1548288,
            "parents": [
                  "fb74c17fad2458f6d8f1f0d3f4dfa7b9ce22a2ec"
            ],
            "uploadDate": "2017-01-10 06:03:52.720000",
            "object_class": "binary.pe32",
            "children": [
                  {
                        "service_name": "srlUnpacker",
                        "status": "success",
                        "child": "6efb5c849b400af96b993284f6d39746b6bdb2e6"
                  },
                  {
                        "service_name": "srlJuice",
                        "status": "success",
                        "child": null
                  },
                  {
                        "service_name": "srlSimilarity",
                        "status": "success",
                        "child": null
                  }
            ],
            "md5": "a7fd6503d8bc6d0e61b779d968568067"
      },
      "message": "Object found",
      "statuscode": 0
}

The status of the various analyses provided by MAGIC are present in the children field. A pending analysis will not have any entry in this field, a failed analysis will have its status field set to “failure” or “failed”, and a successful analysis will have its status set to success.

This file has been successfully unpacked, the genomic information has been successfully extracted, and the magic report correlations are ready. The unpacked version of this file has a SHA1 of 6efb5c849b400af96b993284f6d39746b6bdb2e6. To get the status of the unpacked file, an additional query would have to be made.

For a listing of all services and the analysis they perform, see Services Provided.

vbclient

vbclient -a status 2d9035b27ce5c1cce2fb8432c76e315a1a2a8de0

sha1sum,object_class,uploadDate,UnpackerStatus,JuiceStatus,SimilarityStatus
2d9035b27ce5c1cce2fb8432c76e315a1a2a8de0,binary.pe32,2017-01-10 06:03:52.720000,success,success,success
6efb5c849b400af96b993284f6d39746b6bdb2e6,binary.unpacked,2017-01-10 06:50:23.575000,n/a,success,success

The -a status command for vbclient returns a csv listing the status of the query binary as well as all children of that binary. Each service lists it’s status as success, failure, Pending, or n/a. A status of n/a means that the service will not be run for the file.

This file has been successfully unpacked, the genomic information has been successfully extracted, and the magic report correlations are ready. The unpacked version of this file has a SHA1 of 6efb5c849b400af96b993284f6d39746b6bdb2e6. The status of this unpacked file is automatically queried for and displayed as well. The unpacked file is not going to be unpacked (because it is already unpacked), its genomic features have been extracted, and the magic report correlations are ready.

For a listing of all services and the analysis they perform, see Services Provided.

Download unpacked file¶

Note

Downloading files from MAGIC requires additional permissions. Please contact us for these permissions.

Further Details: Downloading Unpacked File

curl

To download the unpacked version of a binary using curl, we need to know the SHA1 of the unpacked version. This hash can be located using the status query described last section. For our example of 2d9035b27ce5c1cce2fb8432c76e315a1a2a8de0, the unpacked version has a SHA1 of 6efb5c849b400af96b993284f6d39746b6bdb2e6.

curl -O https://api.magic.cythereal.com/download/${VIRUSBATTLE_KEY}/6efb5c849b400af96b993284f6d39746b6bdb2e6

This will download the unpacked file to a file named 6efb5c849b400af96b993284f6d39746b6bdb2e6.

vbclient

Downloading the unpacked version of a binary using vbclient can be done using either the SHA1 of the original binary, or the SHA1 of the unpacked binary. Downloading using the SHA1 of the unpacked binary is straightforward:

vbclient -a download 6efb5c849b400af96b993284f6d39746b6bdb2e6 --enable_malware_download

This will download the unpacked file to ./Results/6efb5c849b400af96b993284f6d39746b6bdb2e6.unp.exe. There may be additional files present in ./Results, but the use of these files is deprecated. They will be removed in the future.

The same action can be used to download both the original and unpacked file using the SHA1 of the original file:

vbclient -a download 2d9035b27ce5c1cce2fb8432c76e315a1a2a8de0

This command will result in two files being created in the ./Results directory: 2d9035b27ce5c1cce2fb8432c76e315a1a2a8de0.exe and 6efb5c849b400af96b993284f6d39746b6bdb2e6.unp.exe. Unpacked files will end with the extension .unp.exe while original files will have the extention exe.

To generate a mapping between the SHA1 of the original file and the SHA1 of the unpacked file, use the map command:

vbclient -a map 2d9035b27ce5c1cce2fb8432c76e315a1a2a8de0

This command will result in the file ./Results/vb-srlUnpacker.map with the contents:

2d9035b27ce5c1cce2fb8432c76e315a1a2a8de0,6efb5c849b400af96b993284f6d39746b6bdb2e6

The SHA1 values on the left are from original files while the SHA1 values on the right are for unpacked files.

Get MAGIC Correlations¶

Futher Details: MAGIC Correlations and Categories

To find binaries that “match”, i.e. are similar to our example binary, we retrieve the MAGIC correlations.

curl

curl https://api.magic.cythereal.com/magic/${VIRUSBATTLE_KEY}/2d9035b27ce5c1cce2fb8432c76e315a1a2a8de0

This results in:

{
    "message": {
        "different_packer_similar_payload": [
            "67120136e48d1307470e22c3a26d3a94bb843d36",
            "a527b9996f8f4c403271a7657f2ad66b6b5a95a8",
            "ee073f6a20320d1a1e5cc435596c1ca27edcda5c"
        ],
        "query": {
            "sha1": "2d9035b27ce5c1cce2fb8432c76e315a1a2a8de0"
        },
        "similar_packer_different_payload": [
            "008019b0ebf88cbcc4feab0347e3e823fdb1a372",
            "07d02e9aefdbc0f424df3ac15f2ff0075447a70a",
            "0e32230d9d8fc115a8097533763a1bb9d506b1aa",
            "0f3a6733d4ad682724a99b3d4cc061162a64d375",
            "0fc3f46da56f89f4d3d1ded07d1abf7708e24a81",
        ],
        "similar_packer_similar_payload": [
            "01ab20c6bb66f5e393f53dd7f54bd2d829745a6e",
            "09eeb0b5c965f8118eadf1a4e465886e14274a60",
            "0b6963786d7ea185a78e6f47e14841636f4d9852",
        ],
        "weak_similar": [
            "0f5c62c1032beb536aa549930e37a83ade94a867",
            "11e33b9cd16c6597f6cdb74a5d73016a6436b8f0",
            "12481531df32b84c35c665023ecb4ac8586da1b6",
        ]
    },
    "status": "success",
    "statuscode": 0,
    "vb_version": "VB-0.4"
}

If you are following along with this tutorial, the output you get might contains many more entries. This is because as new binaries are added to our database, additional matches are found and added to the MAGIC report.

The various types of similarity classes in the MAGIC report are described in MAGIC Correlations.

vbclient

Not currently supported. Planned for future release.

View Genomic Features¶

Futher Details: MAGIC Genomic Features

curl

To get the genomic features for the entire binary:

curl https://api.magic.cythereal.com/show/binary/${VIRUSBATTLE_KEY}/2d9035b27ce5c1cce2fb8432c76e315a1a2a8de0

This will output the JSON document described in Genomic Features

To get filter the output to the features in a single procedure with RVA of 0x1000:

curl https://api.magic.cythereal.com/show/proc/${VIRUSBATTLE_KEY}/2d9035b27ce5c1cce2fb8432c76e315a1a2a8de0/0x1000

Unfortunately, the only way to get the list of procedures in a binary is by retrieving the genomic features for the entire binary. This will be addressed in a future API version.

vbclient

For an entire binary.

vbclient -a show 2d9035b27ce5c1cce2fb8432c76e315a1a2a8de0

For a single procedure at RVA 0x1000.

vbclient -a show 2d9035b27ce5c1cce2fb8432c76e315a1a2a8de0/0x1000

Search for similar procedures¶

Futher Details: Finding Similar Procedures

curl

curl https://api.magic.cythereal.com/search/procs/${VIRUSBATTLE_KEY}/2d9035b27ce5c1cce2fb8432c76e315a1a2a8de0/0x1000

This will result in the JSON document described in Similar Procedure Results.

vbclient

vbclient -a search 2d9035b27ce5c1cce2fb8432c76e315a1a2a8de0/0x1000