Multimodules programming mvantabitdefender com 4 Libraries Copyright Bitdefender

Multi-modules programming mvanta@bitdefender. com

4. Libraries Copyright © Bitdefender 2017 -2019 / www. bitdefender. com

Introduction • Library = a separate collection (module) of code and data resources, meant to • The library resources can be subroutines, classes and objects (OO languages), documentation, user manuals, audio and video items etc. . . • Each published resource must be uniquely identifiable • They usually have associated unique names but other methods can also be used (such as numerical identifiers) • For proper reuse, the behavior and interfaces have to be well documented, as the internal implementation details are either unavailable or should be treaded as such! • User-mode applications are supervised by the OS and can only perform some critical operations through the system libraries. Examples of protected resources: • Hardware: keyboard operations, disk (files), network, audio/video etc. . . • Software resources: threads, memory and application management etc. . . be reused across multiple programs or by sub-modules of the same program. Copyright © Bitdefender 2017 -2019 / www. bitdefender. com

Key concepts • Reference solving (or binding/linking) • The process of searching for a resource inside an external file, making its content available to a program and then replacing the reference with its address. Process name Performed at Performed by Resource origin Process consists of static linking executable creation, at link-time a link editor static libraries, object files searching the libraries and object files for the resources by their names, copying their content inside the new program and replacing the references with addresses pointing to their copies. dynamic linking load-time or run-time the program loader (OS) shared libraries loading the file into memory, finding the named resources and then replacing the references with their memory addresses • Object file (. OBJ) • • An object file is the output produced by the assembler (or a compiler) after processing an input translation unit (a source file). The object file consists of an intermediate & format-specific representation of the instructions and variables found in the input file, alongside metadada such as debugging information, comments, symbolic names and symbol tables etc… • • Intermediate = although they can be linked to form programs or libraries, they’re not executable! Format-specific = a compiler, assembler or linker may support multiple file-formats (ways of encoding the content) with their varying pros and cons (differences in debugging features, segmentation models…) Copyright © Bitdefender 2017 -2019 / www. bitdefender. com

Key concepts • Static library (. LIB) and static linking • • A collection of object files packed together in a single file for convenience. Similar content, properties and use-cases to the. OBJ files. • Shared library (. DLL – Dynamic-link library) and dynamic linking • • • Shared = a single file and memory instance of the library can be used across multiple programs, reducing a program’s size and removing the need to duplicate the resources inside each program using them (as the content isn’t embedded/copied inside a program) Dynamic = linked at program start-up or even at run-time, on demand (as opposed to statically-linked when the executable file is being created). The. DLL files are the Microsoft’s implementation of the shared library concept. • All modern OSs have their own shared library implementation (some equivalent of the. DLL files), conceptually similar ( the ‘what’) but different in usage (the ‘how’). • Import libraries (import. LIB) • • Microsoft-specific files that aren’t actually libraries… • • An import. LIB file only contains a description of what resources a. DLL file contains! • Used for dynamic linking, more info about their role / usage later on… Although they’re similarly named and even have the same file extension, they’re different and should not be confused with the static libraries. LIB files! They can be used in combination: a program might import some resources statically, link some more resources at load-time and bind the remaining ones at run-time! Copyright © Bitdefender 2017 -2019 / www. bitdefender. com

Comparative Analysis • Linking: static vs dynamic Linking Advantages Disadvantages When to use static linking / static libraries - results in a monolithic program (a single file), making it easier to install/uninstall - avoids issues with missing or mismatched library resources - increased program size (usually by many times for small programs) - increased waste of main memory, primary and secondary storage space (the same resources are duplicated by many programs) - simple, stand-alone programs - prototype, study, proofof-concept applications dynamic linking / shared libraries - can be performed at runtime, on demand - pluginable programs - modular updates and bugfixes - reduced program size, storage and main memory waste - can link to OS libraries (which are. DLL files…) - complex un/install logic - recommended (over - potential version compatibility issues static linking) in virtually in any scenario - necessary when linking directly to the OS libraries Copyright © Bitdefender 2017 -2019 / www. bitdefender. com

Comparative Analysis • Dynamic linking: load-time vs run-time Advantages Disadvantages When to use loadtime - easier to use (automatically handled by the OS) - early fail: if a resource is missing or incompatible, the sooner the program fails, the better (no chance of data loss) - increased program start-up time, binding all the necessary libraries, all at once, can take significant time - memory waste: the libraries are kept in memory until the program terminates, even if not needed anymore - this is the preferred method unless the program’s start-up takes too long or there’s a need to support run-time modular updates runtime - better program start-up time: a library is loaded and reference solving occurs right when the resources are actually needed - allows replacing or updating modules without restarting the program - improved memory footprint: keep only the relevant libraries loaded - potential risk of data loss if resources are missing or at wrong version as the program will detect the issue only late, at run-time - lower overall performance if the libraries are loaded and unloaded often - only used rarely, when load-time dynamic linking is not fit for some very specific purpose Copyright © Bitdefender 2017 -2019 / www. bitdefender. com

General usage considerations • Static linking • File by file, all source code must be translated to object code beforehand! • • This includes any needed external and/or third-party resources too! More than one programming language and more than one compiler and/or assembler may be involved! In some cases, the resulting object files may already be considered the end result without any additional processing, for example, they can constitute a SDK (Software Development Kit) all by themselves. Once all the source code was assembled/compiled, a link editor will merge all the resources together, solve any dependencies (the intertwined symbol references) and produce an executable, a static or a shared library file. • • The linker must be compatible with the input object and static library file formats used When the output is a shared library or an executable, the file format used has to be compatible with (or “understood” by) the targeted OS • This is one of the main reasons why compiled programs that support multiple OSs usually have different downloads (binaries) for the supported Oss. • If the result is a static library, that library is meant to be used as an input for another link operation and thus, it has to be compatible with (“understood” by) a link editor • When creating static libraries, the linker can also be referred to as a librarian Copyright © Bitdefender 2017 -2019 / www. bitdefender. com

General usage considerations • Load-time dynamic linking • Allows linking an executable file (or shared library!) with one or more shared libraries • • • The executable and the shared libraries must have already been built at this point, implying static linking had to already have occurred as a prerequisite Performed at load-time, before any CPU instruction of the program’s code is executed The OS program loader (in this context known as a dynamic linker) will load the program and each shared library (. DLL files) referenced by the program for load-time linking into the main memory, proceed to match every imported resource of the program against the resources exported by the shared libraries and replace the references with their final (run-time) addresses. • • A shared/dynamic. DLL library file may also depend on resources found in other libraries, just like an executable file does => the OS will automatically repeat (recursively) the process for every shared library to solve their own dependencies too! Both the executable files and the shared libraries contain import tables as metadata • each import table defines resources imported from a single shared. DLL library file • for every requested resource there’s an entry inside the corresponding import table specifying its name • Each shared library contains an export table listing all the names of the resources it exports together with their corresponding file offsets/positions Copyright © Bitdefender 2017 -2019 / www. bitdefender. com

General usage considerations • Run-time dynamic linking • The binding is performed by the OS, on demand, when explicitly requested by the program. • A call to the kernel 32. dll Load. Library procedure is needed to load the (mandatory) shared library into main memory • It returns a “handle” object (an opaque 32 -bit value) that identifies the loaded library and which is needed for subsequent calls to Get. Proc. Address/Free. Library • • Unless the program explicitly unloads it, once loaded, the whole library will be kept in the program’s memory indeterminately: • • The kernel 32. dll Get. Proc. Address function is called, once for each library resource, for finding out (by inspecting the export table) the memory addresses of the said resources • This function is used for any type of resources, not only for procedures! The kernel 32. dll Free. Library subroutine can be called to unload the library from memory (all of the resource addresses the program knew about are invalid once this call is made!) As the run-time linking support is in itself offered through calls to system library functions (kernel 32. dll), load-time dynamic linking with kernel 32. dll is needed for getting access to (at least) Load. Library and Get. Proc. Address! Copyright © Bitdefender 2017 -2019 / www. bitdefender. com

Libraries & linking overview . . . source N source 1 source 2 . . . source N static import obj 1 obj 2 application EXE code + data Loader source 2 obj N Stack static library LIB linker source 1 obj 2 librarian source N obj 1 uninitialized data obj N dynamic import at load-time obj 1 obj 2 obj N linker . . . compiler / assembler source 2 compiler / assembler source 1 compiler / assembler • The link-editing workflow import library LIB (COFF only) dynamic library DLL module-definitions file DEF (COFF only) Copyright © Bitdefender 2017 -2019 / www. bitdefender. com DLL. . . dynamic, run-time import