Introduction

Two device detection methods are available with the C solution.

Pattern is the same matching method used by our .NET, Java and PHP APIs. However all the necessary data, regular expressions and other pattern recognition methods are provided in the 51Degrees.mobi.c source code file. This results in a faster implementation of the method requiring very little working memory. The pattern method is ideally suited to real time device detection for web sites. As all the detection logic and data is compiled into the executable a recompilation will be required when updates are provided. Therefore the optimal implementation is via dynamic rather than static linking.

Trie indexes require a significant amount of data and are therefore less memory efficient than the pattern matching method. However they are exceptionally fast achieving millions of detections per second. The trie method is suitable for offline analysis of log files or web sites receiving millions of unique requests per minute from an extremely large range of devices.

Our C solution contains source code and data for each of these methods licenced under the AGPL. All our Premium data licences allow the commercial use of this software with our Premium data.

This solution includes the following programs.

  • Command line process which takes a user agent via stdin and returns a device id, or other properties, via stdout.
  • Command line performance evaluation program which takes a file of user agents and returns a performance score measured in detections per second per CPU.
  • A windows only multi-threaded performance evaluation program.
  • A windows example web site which uses the 2 detection methods and compares the results side by side for the requesting browser.

The following section provides a quick start guide to the first item. Subsequent sections explain the performance evaluation programs, and use in a demonstration web site.

Quick Start

Both the Pattern and Trie detection methods can be used via the standard streams stdin and stdout. The following instructions explain how to download the required project files and then build programs which take input and return output via the standard streams.

Pre-requisites

  • One of the following compilers:
    • Visual Studio 2010 or greater
    • GCC compiler for Windows from either MinGW or Cygwin (Ensure your PATH variable is configured so GCC can be called from the command line)
    • Linux environment configured with the GCC compiler
    • Another C99 standards compliant C compiler
  • Download the latest 51Degrees.mobi-Lite-*.C.zip from SourceForge. Decompress into a working folder.
  • If you wish to use the Trie routine download the latest 51Degrees.mobi-Lite-*.trie.zip from SourceForge. Decompress the file into the data folder.

The above Lite files contain free device data properties such as IsMobile, supported browser technologies and screen size in pixels. If you require weekly updates and rich device data download the Premium versions of the above files from the device data download page after purchasing a licence key.

Trie Data File Note - The Lite version of trie data provides 97% matching accuracy. The Premium version provides more than 99% accuracy and has a much larger file size. Premium trie data will require 64 bit versions of the following programs.

Once you've extracted the zip file use the following steps to build the executable for your platform.

Windows

  • Visual Studio
    • Locate the batch file Win32build.bat or Win64build.bat in the root folder.
    • Start the appropriate batch file for the target architecture required.
    • Wait for the batch file to complete compilation. This may take a 5 to 10 minutes.
  • GCC Compiler
    • Locate the batch file WinGCCbuild.bat in the root folder.
    • Start the batch file and wait for it to complete compilation. This may take a few minutes.
  • You will now have executable files called ProcPat.exe and ProcTrie.exe in the root folder. The Perf prefixed files are explained in the Performance section.
  • Execute ProcPat.exe and enter some random text followed by return once it's started. An Id will be returned. Terminate the process by entering an empty string followed by return.

Linux

  • Navigate to the root folder from a terminal session.
  • Type make followed by return. The makefile contained in the root folder will be used to build the executable.
  • You will now have executable files called ProcPat and ProcTrie in the root folder. The Perf prefixed files are explained in the Performance section.
  • Execute ProcPat by typing ./ProcPat. Then enter random text followed by return once it's started. An Id will be returned. Terminate the process by entering an empty string followed by return.

ProcPat

ProcPat uses the pattern matching method to take a UserAgent via stdin, identify properties associated with the UserAgent, and returned them in a pipe (|) separated format. For example the following input would result in the Id of a NokiaN95 being provided.

1
2
NokiaN95
Id|12004-5890-3439-18092

To exit the program press return without entering any text.

To retrieve other properties add them as arguments separated by commas. For example; to return the property IsMobile and ScreenWidthPixels the following command would be used.

1
ProcPat IsMobile,ScreenPixelsWidth

When the UserAgent NokiaN95 is entered these property's values are returned.

1
2
3
NokiaN95
ScreenPixelsWidth|240
IsMobile|True

ProcTrie

ProcTrie works in the same way as ProcPat how ever one additional mandatory command line argument is required. All the patterns are compiled into the ProcPat executable. ProcTrie requires a separate data file containing the trie before it can detect UserAgents.

Uncompress the the Trie zip file downloaded earlier and place the resultant data file in the root folder.

Append the name of the data file to the ProcTrie command as the first argument.

1
ProcTrie 51Degrees.mobi-Lite-2012-12-01.trie.dat

The same interface as the one described in ProcPat is used to receive UserAgent strings and return properties.

The 2nd arguement can contain the required properties separated by commas.

Demonstration Web Site

The main C download includes an example IIS .NET web site ready to run from within Visual Studio 2010 or greater. It includes the following:

  • Dynamic Link Library (DLL) projects for the Pattern and Trie methods exposing required methods suitable for calling from Common Language Runtime.
  • An inter operation wrapper for the two methods exposing functionality via .NET classes. This layer avoids having to make DLL calls directly from your code.
  • A simple web site with one page to display properties using both methods for the requesting browser.

Start by opening the FiftyOne.Mobile.Detection.Provider.sln solution from within the windows folder. Open the Demo web project and locate the Default.aspx.cs file. Change the following line so that it points to the location of your Trie data file.

1
private const string TRIE_DATA_FILE = "..\\..\\51Degrees.mobi.data.dat";

The web site will run within IIS Express and can be started without further changes. Initial compilation may take 5 to 10 minutes as the Pattern DLL is generated. The initial page of the web site when displayed in Chrome emulating a Nexus S should be similar to the following.

Web Page

The Default.aspx.cs page contains the code which creates detection providers, and calls them to determine properties for the device. The following code is relevant.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
private const string TRIE_DATA_FILE = "..\\..\\51Degrees.mobi.data.dat";

        // Initialise the pattern provider with a list of 4 properties.
        private static readonly PatternWrapper _pattern = new PatternWrapper(new[] {
                "Id", "IsMobile", "ScreenPixelsWidth", "ScreenPixelsHeight" });

        // Initialise the tree provider with a data file and a list of 4 properties.
        private static readonly TrieWrapper _trie = new TrieWrapper(
            Path.Combine(AppDomain.CurrentDomain.BaseDirectory, TRIE_DATA_FILE),
            new[] { "Id", "IsMobile", "ScreenPixelsWidth", "ScreenPixelsHeight" });

        // IMPORTANT: For a full list of properties see: 
        // http://51degrees.mobi/products/devicedata/propertydictionary

        protected void Page_Init(object sender, EventArgs e)
        {
            // Get properties for the current useragent.
            var patternProperties = _pattern.GetProperties(Request.UserAgent);
            var trieProperties = _trie.GetProperties(Request.UserAgent);

            // Output the properties from each provider.
            var builder = new StringBuilder();
            builder.Append("<table>");
            builder.Append("<tr><th></th><th>Pattern</th><th>Trie</th></tr>");
            foreach (var property in patternProperties)
            {
                builder.Append("<tr>");
                builder.Append(String.Format(
                    "<td><b>{0}</b></td>",
                    property.Key));
                builder.Append(String.Format(
                    "<td>{0}</td>",
                    String.Join(",", property.Value)));

                // Add the tree property if one exists.
                if (trieProperties.ContainsKey(property.Key))
                    builder.Append(String.Format(
                        "<td>{0}</td>",
                        String.Join(",", trieProperties[property.Key])));
                else
                    builder.Append("<td></td>");
                builder.Append("</tr>");
            }
            builder.Append("</table>");

            Results.Text = builder.ToString();
        }

The following lines warrant a little extra explanation.

Line 4 and 5 construct a Pattern provider initialised to return the 4 properties Id, IsMobile, ScreenPixelsWidth and ScreenPixelsHeight. Notice the provider is static so that it can be shared across multiple instances of the page. If the provider were required on many pages then it would be better to place it in a static class, or in the global.asax file.

Lines 8 to 10 construct a Trie provider. The constructor is the similar to the Pattern provider but also requires the location of the Trie data file.

Line 18 calls the GetProperties method of the Pattern provider to return a List where the key is the property string used at construction and the value is a list of values for the property. Most properties return only one value in the list, however some like CcppAccept may return many. See the Property Dictionary for more information.

Line 19 returns a List in the same structure as line 18 using the Trie provider as it's data source.

The remaining lines then produce a HTML table to display the property values returned for each provider. Line 33 joins the list of values into a single comma separated string.

Interop

The FiftyOne.Mobile.Detection.Provider.Interop.csproj project contains the wrapper classes used to call the native DLLs from CLR managed code. They handle all the complexity associated with multi-threading, initialising the DLL and freeing memory when the wrapper instance is destroyed.

This example shows them being used in a web site, but they could equally be used for offline analysis or as part of a web service.

The main advantage using the Interop layer with the Pattern matching method has over our entirely native .NET implementation is lower memory use as all the working data is compiled into the DLL as ASCII strings. The matching performance is also marginally faster.

If performance is essential and memory consumption not a factor the Trie matching method is the best solution.

Importantly these solutions will only work in a .NET application or web site configured with full trust.


Common Challenges

64 Bit & Trie

Most 32 bit systems we've tested with can not allocate more than 2 GB of continuous memory. If the Trie format data file is larger than 2 GB (Premium version) then it can not be loaded into a 32 bit application. Therefore 64 bit will need to be used.

When using a 64 bit configuration with IIS either IIS Express 8 or above will need to be installed, or a full version of IIS 7 or above will be required. If an error message similar to the following is displayed after the web site is loaded "Could not load file or assembly … An attempt was made to load a program with an incorrect format (System.BadImageFormatException)" the most likely cause will be trying to run 64 bit DLLs in a 32 bit IIS.

IIS 32 bit & 64 bit

The default demonstration web site solution compiles in 32 bit and is configured to start in IIS Express. If the configuration is changed to 64 bit IIS Express 7 can not be used as it does not support 64 bit executable. Therefore the option to use IIS Express (or the Cassini development server) is removed and the local IIS machine will need to be used. When moving to local IIS ensure that the Application Pools identity has sufficient privileges to read from the web site folder, and also if used the trie format data file.

Performance Evaluation

2 performance tests are included with the C library for single and multi-threaded evaluation. Single threaded performance evaluation will work on any platform, multi-threaded has been written exclusively for Windows.

Single Threaded - All Platforms

The build process described in the Quick Start section also produced 2 additional executable called PerfPat and PerfTrie. Each uses the pattern and trie detection methods respectively. They both read an input file of user agents, detect the corresponding devices and then finally return an average detection time.

PerfPat

Runs a performance test using a single process and thread using a data file of user agent strings as input. Uses the pattern matching detection routine which is very memory efficient.

PerfPat.exe UserAgentsFile [Properties]

UserAgentsFile

Path to a list of user agents to be used in the performance test.

http://51Degrees.mobi/Million.zip contains a test file of one million user agents.

[Properties]

A comma separated list of properties to be returned. If not provided Id will be used.

For example: Id,IsMobile will return the Id of the matched device and True or False to indicate if the device is a mobile.

Examples: PerfPat.exe million.csv
PerfPat.exe million.csv Id,IsMobile

PerfTrie

Runs a performance test using a single process and thread using a data file of user agent strings as input. Uses the trie matching detection routine which is very fast, but uses a lot of memory.

PerfTrie.exe TrieDataFile UserAgentsFile [Properties]

TrieDataFile

A source data file in trie format. See the readme.txt file in the data folder for more details.

UserAgentsFile

Path to a list of user agents to be used in the performance test.

http://51Degrees.mobi/Million.zip contains a test file of one million user agents.

[Properties]

A comma separated list of properties to be returned. If not provided Id will be used.

For example: Id,IsMobile will return the Id of the matched device and True or False to indicate if the device is a mobile.

Examples: PerfTrie.exe 51Degrees.mobi.trie.dat million.csv
PerfTrie.exe 51Degrees.mobi.trie.dat million.csv Id,IsMobile

The following screen shot demonstrates the results for PerfTrie.

Multi Threaded - Trie & Windows

A C++ windows project is included to launch a multi-threaded version of the performance evaluation using the trie method only. The executable uses the same command line arguments as those used with PerfTrie.

Start by opening the FiftyOne.Mobile.Detection.Performance.sln solution from within the windows folder. Ensure the Command Arguements are configured to provide the correct path to the Trie data file, and the data file containing the test user agents. The following screen shows the correct configuration for files located in the data folder of the project.

Reference

This reference guide describes the external C functions. Functions not documented here should either not be used as their intended for internal purposes or are not yet ready for external use.

Pattern

The pattern method library uses worksets to reference any working memory or parameters needed to perform detection. Worksets are created following the initialisation of the library and are then used by subsequent functions. The library can be initialised once, many worksets can be created and destroyed. When all worksets have been freed the library can be destroyed. See the ProcPat.c file for an example of how to use these methods.

Method Parameter Description
init Initialises the library with properties to return from each detection. A full list of properties is available in the property dictionary.
properties Pointer to a comma separated list of properties to return.
destroy Frees any memory used. Must be called after init and after all worksets have been freed.
createWorkset Returns a pointer to a workset structure used in subsequent calls.
freeWorkset Frees the memory used by the workset provided.
ws Pointer to a workset.
getDevice Returns a pointer to a device which matches the user agent provided in the input field of the workset structure.
deviceOffset The integer value returned from getDeviceOffset.
ws Pointer to a workset whose input field points to the user agent of the device to be matched.
processDeviceCSV

Sets the provided results character array to a CSV format string using pipe (|) characters to separate fields. The number of rows returned will be equal to the number of valid properties provided to the init function. The function returns the length of the result array used.

Example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
char output[4096];
const Device *device;

gets(ws->input);
device = getDevice(ws);

if (device != NULL) {
    processDeviceCSV(device, output, 4096);
    printf("%s", output);
}

The following lines from the above example are relevant.

Line 4 sets the worksets input field to point to a useragent string captured from stdin.

Line 5 uses getDevice to return a pointer to a device matching the input useragent string.

Line 8 uses processDeviceCSV to return a pipe seperated list of properties for the device returned.

subject Pointer to a device returned from getDevice.
result Pointer to a preallocated array of characters to hold the result.
resultLength The length of the array of characters available to hold the result.

Trie

The trie matching method is computationally very simple and as such does not require any working memory for detection requests. See the ProcTrie.c file for an example of how to use these methods.

Method Parameter Description
init Initialises the library with a data source and properties to return for each detection. A full list of properties is available in the property dictionary.
fileName Pointer to the relative of absolute path to the trie format data file.
properties Pointer to a comma separated list of properties to return.
destroy Frees any memory used. Must be called after init when the library has been finished with.
getDeviceOffset Returns the integer offset in the array of devices for the device which matches the useragent provided.
userAgent Pointer to a useragent string of the device being requested.
getPropertyIndex Returns the integer index of for the property requested, or -1 if the property is not available. A full list of properties is available in the property dictionary.
value Pointer to a property name.
getValue Returns a pointer to a string containing the value for the requested device and property.
deviceOffset The integer value returned from getDeviceOffset.
propertyIndex The integer value returned from getPropertyIndex.
processDeviceCSV

Sets the provided results character array to a CSV format string using pipe (|) characters to separate fields. The number of rows returned will be equal to the number of valid properties provided to the init function. The function returns the length of the result array used.

Example

1
2
3
4
char input[50000], output[50000];
gets(input);
processDeviceCSV(getDeviceOffset(input), output, 50000);
printf("%s", output);

Line 2 gets a user agent string from stdin.

Line 3 gets the offset to the device and passes it directly to the processDeviceCSV along with the output buffer and the length of the buffer.

deviceOffset The integer value returned from getDeviceOffset.
result Pointer to a preallocated array of characters to hold the result.
resultLength The length of the array of characters available to hold the result.
Pre-Compilation - Building pcre_chartables.c

The PCRE library used to process regular expressions has to be configured in a pre-compilation stage to build character tables for the target architecture. There source code file pcre_chartables.c is not distributed with the project. All the batch files, make files, and Visual Studio projects provided with the download include this pre-compilation activity.

The following lines from the batch files are relevant to this activity.

  1. if exist pcre_chartables.c del pcre_chartables.c - If the character tables are already present they are deleted.
  2. if exist dftables.c cl /w dftables.c - If the C source code needed to create the executable that will then build the character tables is present then build it.
  3. if exist dftables.exe dftables.exe pcre_chartables.c - If the executable was created use it to create the character tables source code file.
  4. if exist dftables.exe del dftables.exe - Delete the executable used to create the character tables.
  5. if exist dftables.obj del dftables.obj - Delete the object file used in compilation.

The pcre_chartables.c file has to be built using dftables.c before the main library compilation can begin.

Premium Data

What is Premium Data?

The 51degrees.mobi_Detection_C.zip file comes with the Lite data set embedded in the header file and will use it by default. While using the Lite data set, only the essential information about the device connecting to your website will be returned. For example, whether it is mobile or not and the screen resolution.

If you require more information about the device connecting to your site then the premium data set will be able to provide a more detailed description, such as the hardware model, the operating system being run, the browser being used etc. For a full list of properties please see our premium device properties page and for more information about the premium data, see the device data page.

Using Premium Data

If you decide to purchase Premium data set then you will be provided with a premium data zip file similar to the lite data outlined above. The results string returned from using this data set will be much larger and more detailed, allowing the end user to parse the string for more data than provided by the lite set.