Querying Analysis Status

API Endpoints

GET /query/(api_key)/(sha1)

https://api.magic.cythereal.com/docs#!/query/Query_Info

Query various information, including analysis status, about file with SHA1 equal to (sha1). See Results Description for description of the output of this endpoint.

CLI Commands

Note

This sections assumes knowledge from the sections Results Description, File Object Classes, and Services Provided. Specifically, this sections assumes you know what object classes, services, and file children are.

status

vbclient -a status [--norecursive] SHA1 ...

The -a status action reports the analysis status for all query files as well as for all children of the query binaries. Thus, if the file indicated by SHA1 is an archive file, all files extracted from the archive will have their analysis status reported. Only files with an object class of binary.* have their status reported. The option --norecursive disables reporting of the status of children.

The output of the -a status command is a CSV listing for each service type whether the analysis was a success, failed, is pending, or will not be run (n/a). An example is given below.

$ vbclient -a status 300489be43b6b8d95d2e0c5196885a0754f4adaf
sha1sum,object_class,uploadDate,UnpackerStatus,JuiceStatus,SimilarityStatus
0f5c62c1032beb536aa549930e37a83ade94a867,binary.pe32,2017-03-22 15:50:14.957000,success,success,success
f946360fa31b5a0ba3261a1c5be0b601b757cd3f,binary.unpacked,2017-03-22 15:51:14.450000,n/a,success,success
13ba811b3719803bb28b3bf6616a3192f0838ae5,binary.pe32,2017-03-22 15:50:15.045000,success,success,success
a5a615cbc13e77f9129e3bbf70dde30a45eab815,binary.unpacked,2017-03-22 16:13:11.953000,n/a,success,success
2d9035b27ce5c1cce2fb8432c76e315a1a2a8de0,binary.pe32,2017-03-22 15:50:15.092000,success,success,success
6243a9a92c394dcd86f8f37ec020bdd5a503e1b1,binary.unpacked,2017-03-22 15:56:41.353000,n/a,success,success
41a5aa4588c7ca21d730845e0f4b2a26fdedd8dc,binary.pe32,2017-03-22 15:50:15.134000,success,success,Pending
ccf12ae854830481bae880fa852272a11bd114ae,binary.unpacked,2017-03-22 15:58:57.951000,n/a,success,Pending

query

vbclient -a query [--norecursive]  SHA1 ...

The -a query action returns the same information as the query API Endpoint. The main difference is that the CLI command will by default call the query command recursively for all children of the query binaries. See Results Description for a description of what the children a file are. The recursive query can be disabled by providing the --norecursive option.

Results Description

Warning

Any fields not documented here should be considered deprecated. Such fields may be removed in the future without warning.

There are two pieces of information returned by the query endpoint:

• Information about the queried file.
• Results of the analysis performed on the file.

An example query result is given below for the malware with SHA1 of 2d9035b27ce5c1cce2fb8432c76e315a1a2a8de0.

{
    "sha1": "2d9035b27ce5c1cce2fb8432c76e315a1a2a8de0",
    "md5": "a7fd6503d8bc6d0e61b779d968568067",
    "uploadDate": "2017-01-10 06:03:52.720000",
    "unix_filetype": "PE32 executable (GUI) Intel 80386, for MS Windows",
    "origFilepath": [
      "./archive"
    ],
    "object_class": "binary.pe32",
    "parents": [
      "fb74c17fad2458f6d8f1f0d3f4dfa7b9ce22a2ec"
    ],
    "children": [
      {
        "service_name": "srlUnpacker",
        "status": "success",
        "child": "6efb5c849b400af96b993284f6d39746b6bdb2e6"
      },
      {
        "service_name": "srlJuice",
        "status": "success",
        "child": null
      },
      {
        "service_name": "srlSimilarity",
        "status": "success",
        "child": null
      }
    ],
}

All but the last two fields provide information about the queried file. The fields sha1 and md5 contain the respective cryptographic hash of the queried file. The uploadDate field records when the file was originally uploaded to MAGIC and unix_filetype contains the result of the Unix file command when run on the queried file. The origFilepath field contains the file path of the file when it was uploaded to MAGIC. Finally, the object_class field records the object class that MAGIC has placed the file in. These object classes are documented in File Object Classes.

The results of any analysis or operation performed on a file is represented through parent-child relationships between the analyzed file and the result of the analysis. These relationships are captured by the parent and children fields. When a particular analysis completes for a given file, the results of that analysis are added to the list in the children while the SHA1 of the analyzed file is added to the parent list of any newly created files.

Each entry in the children list is a dictionary with the keys service_name, status, and child. The service_name field lists the type of analysis or action performed on the file. The full list of services is documented at Services Provided. The status field records if the analysis was a “success” or a “failure”. Finally, the child field contains the SHA1 of any file created as a result of this analysis or action. If no file resulted from the analysis, this field is null.

If a single type of analysis or action results in multiple files being created, multiple entries in the children list will be created. Each of these entries will contain the same value for service_name.

The contents of children will differ depending on what the object_class (File Object Classes) of the file is. Binary files, i.e. object_class equal to binary.*, will contain a list of the analysis results for the binary. Archive files, i.e. object_class equal to archive.*, will contain a list of files extracted from the archive. Each entry in the children list will have service_name equal to srlArchive and contain the SHA1 of the extracted file.

An example of the children for an archive file is given below:

{
    "object_class": "archive.gz",
    "children": [
        {
            "service_name": "srlArchive",
            "status": "success",
            "child": "2d9035b27ce5c1cce2fb8432c76e315a1a2a8de0"
        },
        {
            "service_name": "srlArchive",
            "status": "success",
            "child": "949728bb9ec8a5032ab89201932260e28cf73b7d"
        }
    ],
}

File Object Classes

Warning

Any object class not documented here should be considered deprecated. Such object classes may be removed in the future without warning.

The object class of a file is the type of object MAGIC identified the file as. This object class file determines which services, i.e., types of analysis or processes, that will be run on a file. The object classes defined in MAGIC are documented in the table below. The services that will run for any given object class are documented in Services Provided.

Object Class	Description
archive.7z	7z compressed archive
archive.tar	tar archive
archive.tgz	gzip compressed archive
archive.zip	zip compressed archive
binary.pe32	Windows 32 PE executable
binary.elf	An ELF executable
binary.unpacked	An unpacked binary

The following diagram summarizes the various classes of object classes and their parent-child relations.

The Archive class represents the variety of compressed formats, such as, zip, tar, etc. An Archive object may contain another Archive object or an Executable. The ‘Executable’ class represents the original executable files contained in an Archive. The result of unpacking an Executable is an object of class Unpacked.

The diagram below presents a more detailed view of how the various file types are related.

Services Provided

Warning

Any service not documented here should be considered deprecated. Such services may be removed in the future without warning.

The services, i.e. analysis or processes, provided by MAGIC are documented in the table below. This table lists the name of the service as it will appear in the children list documented in Results Description, the types of object classes the service will be run on, the types of object classes the service will produce, and a description of what the service does. Object classes listed as something.* are interpreted as “all object classes that start with substring something.”.

service_name	Operates On	Produces	Description
srlArchive	archive.*	Various	Extracts files from an uploaded archive file. Recursively submits all extracted files to MAGIC
srlUnpacker	binary.*	binary.unpacked	Unpacks binary files.
srlJuice	binary.*	null	Extracts the genomic features from binary files.
srlSimilarity	binary.*	null	Performs the calculations necessary for the MAGIC Report correlation