Ask Better Questions About Your Network With Nmap

Ask Better Questions About Your Network With Nmap Scripting Engine Educause Security Professionals Conference, May 2019

Ask Better Questions About Your Network With Nmap Scripting Engine Setup (Optional) ● Download files at https: //uchicago. box. com/v/educause-spc-nmap ● Install Nmap on your system: https: //nmap. org/download. html ● Verify Nmap install by printing the version number $ nmap -V OR C: Program Files (x 86)Nmap> nmap. exe -V

$ whoami James Clark ● current: Director of Information Security ● previous: security engineer, incident responder, system administrator, developer

Agenda and Goals Why? show you how to translate questions about your network into scripts that use full power of Nmap so that you can get immediate, actionable results How? mix conceptual and practical; optionally hands-on What? show range of uses; demonstrate how to use, how to customize

Nmap $ man 1 nmap NAME nmap — Network exploration tool and security / port scanner SYNOPSIS nmap [Scan Type. . . ] [Options] {target specification}

vs demos examples

Example: Default Scan $ nmap 45. 33. 32. 156 Starting Nmap 7. 60 ( https: //nmap. org ) at 2019 -mm-dd CDT Nmap scan report for scanme. nmap. org (45. 33. 32. 156) Host is up (0. 067 s latency). Not shown: 991 closed ports PORT STATE SERVICE 22/tcp open ssh 25/tcp filtered smtp 80/tcp open http 9929/tcp open nping-echo 31337/tcp open Elite Nmap done: 1 IP address (1 host up) scanned in 2. 27 seconds

Example: Version Detection, OS Fingerprinting $ sudo nmap -s. V -O scanme. nmap. org PORT STATE SERVICE VERSION 22/tcp open ssh Open. SSH 6. 6. 1 p 1 Ubuntu (Ubuntu Linux; protocol 2. 0) 25/tcp filtered smtp 80/tcp open http Apache httpd 2. 4. 7 ((Ubuntu)) 9929/tcp open nping-echo Nping echo 31337/tcp open tcpwrapped Aggressive OS guesses: Linux 3. 10 - 4. 8 (98%), Linux 3. 16 - 4. 6 (96%), HP P 2000 G 3 NAS (93%) No exact OS matches for host (test conditions non-ideal). Service Info: OS: Linux; CPE: cpe: /o: linux_kernel OS and Service detection performed. Please report any incorrect results at https: //nmap. org/submit/. Nmap done: 1 IP address (1 host up) scanned in 26. 84 seconds

Example: Default Scripts $ sudo nmap -s. V -O --script=default scanme. nmap. org PORT STATE SERVICE VERSION 22/tcp open ssh Open. SSH 6. 6. 1 p 1 Ubuntu 2 ubuntu 2. 11 (Ubuntu Linux; protocol 2. 0) | ssh-hostkey: | 1024 ac: 00: a 0: 1 a: 82: ff: cc: 55: 99: dc: 67: 2 b: 34: 97: 6 b: 75 (DSA) | 2048 20: 3 d: 2 d: 44: 62: 2 a: b 0: 5 a: 9 d: b 5: b 3: 05: 14: c 2: a 6: b 2 (RSA) | 256 96: 02: bb: 5 e: 57: 54: 1 c: 4 e: 45: 2 f: 56: 4 c: 4 a: 24: b 2: 57 (ECDSA) |_ 256 33: fa: 91: 0 f: e 0: e 1: 7 b: 1 f: 6 d: 05: a 2: b 0: f 1: 54: 41: 56 (Ed. DSA) 25/tcp filtered smtp 80/tcp open http Apache httpd 2. 4. 7 ((Ubuntu)) |_http-server-header: Apache/2. 4. 7 (Ubuntu) |_http-title: Go ahead and Scan. Me! 9929/tcp open nping-echo Nping echo 31337/tcp open tcpwrapped [snip]

Nmap Scripting Engine (NSE) Lua interpreter plus Nmap API and: ● scripts (596+ in 14 categories) ● libraries (136+) ● data files (32+) Reference: https: //nmap. org/nsedoc

Nmap and NSE Files find Nmap installation directory, e. g. ● /usr/share/nmap ● /usr/local/share/nmap ● C: Program Files (x 86)Nmap ● scripts (scripts and script. db) ● nselib (libraries) ● nselib/data (data files)

Example: Running Scripts $ nmap --script=<scripts> can be any of: boolean/wildcard expression, script categories, script file names, script directory names 1 $ nmap -p 80 --script http-generator. nse 2 $ nmap -p 443 --script=http-headers, http-title, http-generator 3 $ nmap --script="default and http-*"

Example: Script Arguments --script-args = “scriptname. arg 1=value, scriptname. arg 2=value” Arguments are name-value pairs provided to relevant scripts. $ nmap --script dns-fuzz, dns-brute --script-args dns-fuzz. timelimit=1, dns-brute. threads=2 --script-args-file = filename alternative: provide arguments in a file $ nmap --script dns-fuzz, dns-brute --script-args-file dns-args. txt

Example: Getting Help for Scripts (and Finding Scripts) --script-help=<scripts> (expression, category, file name, directory name) $ nmap --script-help http-vuln-cve 2017 -1001000 Categories: vuln safe https: //nmap. org/nsedoc/scripts/http-vuln-cve 2017 -1001000. html Attempts to detect a privilege escalation vulnerability in Wordpress 4. 7. 0 and 4. 7. 1 that allows unauthenticated users to inject content in posts. . --script-trace (append to your nmap command)

Demo!: Using a Script Type this. Watch for spaces in script names. Running against your own site may be more interesting. 1$ nmap -p 443 --script=http-headers, http-title, http-generator security. uchicago. edu . . . |_http-generator: University of Chicago Blogs |_http-title: Information Security | Helping to secure the UChicago community 2 $ nmap -p 443 --script=http-headers --script-args http. useragent=netcat security. uchicago. edu

Demo: Expressions for Getting Help (and Finding Scripts) $ nmap --script-help safe, http* $ nmap --script-help "safe and http*" $ nmap --script-help "safe and http* and discovery and not vuln" | grep ^http-affiliate-id https: //nmap. org/nsedoc/scripts/http-affiliate-id. html. . . C: Program Files (x 86)Nmap> nmap. exe --script-help "mysql*" | findstr /b [m. C] mysql-audit Categories: discovery safe mysql-brute Categories: intrusive brute. . .

Demo: NSE Script Trace Rerun earlier scripts with --script-trace appended, or choose a DNS or SMTP script example. Substitute your domain or SMTP server. Use a paging command like more to step through output. C: Program Files (x 86)Nmap> nmap --script dns-srv-enum --script-args "dns-srvenum. domain=opendns. com" --script-trace | more C: Program Files (x 86)Nmap> nmap -Pn -p 465, 587 --script smtp-commands smtp. gmail. com --script-trace | more

Enough Lua to be Dangerous: Intro

Enough Lua to be Dangerous: Tables port “number” 3306 “protocol” “tcp” “state” “open” “version” table: 0 x 23 FE 00 ● ● a table is a collection of key-value pairs Lua’s only compound data structure used for (almost) everything can contain (almost) anything port. version “name” "mysql" “name_confidence” 3 “cpe” “cpe: /a: mysql ”

Enough Lua to be Dangerous: Hands On https: //repl. it/repls/folder/NSE

Enough Lua to be Dangerous: How Tables are Represented port “number” 3306 “protocol” “tcp” “state” “open” “version” table: 0023 FE 00 port. version “name” "mysql" “name_confidence” 3 “cpe” “cpe: /a: mysql” port = { number=3306, protocol=”tcp”, state=”open”, version={ name=”mysql”, name_confidence=3, cpe=”cpe: /a: mysql” } }

Demo: simple. nse -- HEAD provides metadata description = "Prints a friendly message if a particular port is open. " categories = {"safe"} -- RULE called for every open port on every host. receives tables with target's host and -- port information. returns true or false to decide "should we call the action function? " portrule = function(host, port) return (port. number == 443) end -- ACTION called when rule returns true. does the actual work of the script -- then returns table with results to Nmap action = function(host, port) return {message = "Ola Mundo"} end

Demo: Simple Script (Output) $ nmap -p 443 --script=simple. nse google. com PORT STATE SERVICE 443/tcp open https | simple: |_ message: Ola Mundo $ nmap -p 80, 443 --script=simple. nse google. com PORT STATE SERVICE 80/tcp open http 443/tcp open https | simple: |_ message: Ola Mundo

Nmap Programming Interface host port “ip” 10. 1. 1. 1 “number” 3306 “name” “dbserver” “protocol” “tcp” “os” table: 0 xnn nmap. registry --script-args “args” table: 0 xnn “example” table: 0 xnn “state” “open” “registry” table: 0 xnn action = function(host, port) local myarg = stdnse. get_script_args(. . . ) local registry_example = nmap. registry. example -- do more stuff, then return results to display return {key 1=”some value”, key 2=”other value”} “key 1” “some value” “key 2” “other value”

NSE Libraries Scripts use libraries to handle protocol communication (e. g. http, smb), access frameworks (e. g. brute, vulns) or helper functions, e. g. ● stdnse: standard NSE functions, e. g. debug, get_script_args ● nmap: Nmap’s internal functions and data structures, e. g. fetchfile, log_write To use a library simply load it in your script using require -- load the standard NSE library local stdnse = require "stdnse" -- call functions from that library local example = stdnse. get_script_args("example")

Example: Head (Metadata) description = [[ The first paragraph should be a brief synopsis of the script function suitable for stand-alone presentation to the user. Further paragraphs may provide much more script detail. ]] ---- @usage -- nmap --script $SCRIPTNAME [--script-args $SCRIPT-ARGS] <target> -- @args $SCRIPT-ARGS -- @output --author = "Your Name" license = "Same as Nmap--See http: //nmap. org/book/man-legal. html" categories = {"safe, discovery"} dependencies = {"http-enum"}

Example: Rule -- example manual port rule for VNC portrule = function(host, port) return (port. number == 5900 or port. number == 5901 or port. number == 5902) and port. service == “vnc” and port. protocol == "tcp" and port. state == "open" end Types of Rules ● prerule ● hostrule ● portrule ● postrule -- alternative: shortport library makes it easier local shortport = require “shortport” -- load the library -- easier version of vnc example portrule = shortport_or_service( {5900, 5901, 5902} , "vnc", "tcp", "open") -- common usage - covers 11 ports and 12 services associated with http portrule = shortport. http

Nmap and NSE Phases (and Rules) 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. Script pre-scanning ← prerule Target enumeration Host discovery Reverse-DNS resolution Port scanning Version detection * OS detection Traceroute Script scanning ← hostrule or portrule Output Script post-scanning ← postrule Types of Rules ● prerule ● hostrule ● portrule ● postrule

Example: Action action = function(host, port) local path = stdnse. get_script_args('http-generator. path') or '/' local pattern = '<meta name=["']? generator["']? content=["']([^"']*)["'] ? /? >' local response = http. get(host, port, path) if ( response and response. body ) then return response. body: match(pattern) end $ nmap -p 443 --script http-generator wordpress. com 443/tcp open https |_http-generator: Word. Press. com

NSE Data Files: Why, Where, and How ● Nmap-provided data files provide a starting point for many use cases but are not the last word and do not cover all use cases - make your own! ● easiest and most common way for you customize NSE is to revise or create new data files ● You can override default data directory (nselib/data) at runtime using --datadir (or sometimes --script-args) ● Data files are typically either simple line-oriented list (e. g. usernames. lst) OR Lua tables (e. g. http-fingerprints. lua)

Example NSE Data File: Excerpt from http-fingerprints. lua fingerprints = {}; table. insert(fingerprints, { category = 'general', probes = { { path = '/console/login. Form. jsp', method = 'HEAD' } }, matches = { { match = '', output = 'Oracle Web. Logic Server Administration Console' } } });

Example: Using Custom NSE Data Files -- http-enum. nse uses http-fingerprints. lua unless new http-enum. fingerprintfile requested local fingerprint_file = stdnse. get_script_args('http-enum. fingerprintfile') or 'http-fingerprints. lua' $ nmap -p 443 --script http-enum --script-args httpenum. fingerprintfile=weblogic. lua 10. 0/8 Nmap scan report for example (10. 10. 10) PORT STATE SERVICE 443/tcp open https | http-enum: |_ /console/login/Login. Form. jsp: Oracle Web. Logic Server Administrative Console

Using Unofficial Scripts or Data Files in Production Official Nmap Warning “Scripts are not run in a sandbox and thus could accidentally or maliciously damage your system or invade your privacy. Never run scripts from third parties unless you trust the authors or have carefully audited the scripts yourself. ” Trusted data files 1. Add to data directory nselib/data (perhaps with naming convention to differentiate from official) 2. Add to alternative data directory and use --datadir Trusted scripts 1. Add to standard scripts directory then update scripts. db (read carefully re: categories) 2. Add to alternative scripts directory and always run with path specified in --script $ sudo nmap --script-updatedb

Demo: Editing an Existing Script, Part I Many sites use the Twitter Card framework to do … something or other. Let’s take advantage of it to get the Twitter handles of the sites we are scanning. There is no time or reason to start from scratch, so let’s find an existing script to revise. . .

Demo: Editing an Existing Script, Part II … our old friend http-generator. nse, renamed to match its new, hip functionality. Download files from Box include: 1. http-getsocial. nse = lightly edited version of http-generator. nse to use as starting point for us to modify - Head and Rule are completed - we have to modify Action 2. http-getsocial-final. nse = example finished product for reference Edit http-getsocial. nse using the embedded instructions. We’re going to modify it to grab two different meta tags.

Ask Better Questions About Your Network With NSE: Summary 1. How NSE fits into overall Nmap architecture and functionality 2. Categories and a handful of examples of divergent use cases for scripts 3. How to run scripts by name, category, directory, expression, etc (with arguments if necessary) 4. How to get help on scripts using local commands or files (or the NSEDOC site) 5. Enough Lua to be dangerous 6. Structure, API, libraries for scripts 7. Customizing script data files 8. Customizing scripts

James Clark <jclark@uchicago. edu>

Extra: Force Script $ nmap -p 3000 --script=+http-headers, http-title --scriptargs="http. useragent=Mozilla/5. 0" 1. 2. 3. 4 PORT STATE SERVICE 3000/tcp open ppp | http-headers: | Content-Type: text/html; charset=UTF-8 | Set-Cookie: grafana_sess=0 f 0731 f 5 f 6423160; Path=/; Http. Only | Date: Sun, 28 Apr 2019 23: 28: 03 GMT | Connection: close | |_ (Request type: HEAD)

Extra: More Script Arguments “A user agent is any software that retrieves, renders and facilitates end user interaction with Web content. ” - World Wide Web Consortium (W 3 C) $ cat /usr/local/share/nmap/script-args. file http. useragent=Mozilla/5. 0 (Windows NT 10. 0; Win 64; x 64) Apple. Web. Kit/537. 36 useget=true $ nmap -p 443 --script=http-security-headers --script-args-file=. /script-args. file w 3 c. org

Extra: Troubleshooting NSE Scripts: Debugging Verbosity (-v, -vv or -v 2, etc OR keypress v at runtime stdnse. verbose(level, format-string, other-arguments) Debugging (-d, -dd or -d 2, etc OR keypress d at runtime) stdnse. debug(level, format-string, other-arguments) stdnse. debug 1(format-string, other-arguments) if current running level >= to given level print the formatted message level is optional numeric first argument used as the level necessary to print the message (it defaults to 1 if omitted) Difference? Verbosity more for user, Debugging more for developer -- example debugging - higher debug levels provide more details stdnse. debug(1, "Loading datafile: %s", filename) stdnse. debug(2, ”Datafile contains %d records”, #records)

Extra: Troubleshooting NSE Scripts: Tracing --version-trace: Show detailed version scan activity --script-trace: Show all data sent and received --packet-trace: Show all packets sent and received

Extra: Example: Script Tracing $ nmap -p 443 --script=http-title, http-generator www. drupal. org PORT STATE SERVICE 443/tcp open https |_http-title: Access to this page has been denied. $ nmap -p 443 --script=http-title, http-generator www. drupal. org --script-trace | tail -20 <!DOCTYPE html><h 1>Access to this page has been denied. </h 1> <p>You have been blocked because we believe you are using automation tools to browse the website. </p> $ nmap -p 443 --script=http-generator --script-args="http. useragent=Mozilla/5. 0" www. drupal. org PORT STATE SERVICE 443/tcp open https |_http-generator: Drupal 7 (https: //www. drupal. org)

Extra: shortport Library for portrules shortport provides functions for building short portrules - just declare what you are looking for and the module will do the work then return true or false -- remember to load the module in your script local shortport = require “shortport” -- example 1: specify the ports, services, protocols, and states you are looking for portrule = shortport_or_service( {5900, 5901, 5902} , "vnc", "tcp", "open") -- example 2: covers common ports and services that run http -- LIKELY_HTTP_PORTS = {80, 443, 631, 7080, 8443, 8088, 5800, 3872, 8180, 8000} -- LIKELY_HTTP_SERVICES = { "http", "https", "ipp", "http-alt", "https-alt", "vnc-http", -"oem-agent", "soap", "http-proxy", "caldav", "carddav", "webdav"} portrule = shortport. http

Extras: Enough Lua to be Dangerous: Advanced Topics that we are not going to cover today but are worthwhile for the next level up. ● ● ● String handling Regex Coroutines Sockets Objects Metatables

Extra: Fingerprint in Table Format table. insert(fingerprints, { name = "Apache Axis 2", category = "web", paths = { {path = "/axis 2 -admin/"} }, login_combos = { {username = "admin", password = "axis 2"} }, login_check = function (host, port, path, user, pass) local resp = http_post_simple(host, port, url. absolute(path, "login"), nil, {user. Name=user, password=pass, submit=" Login "}) return resp. status == 200 and (resp. body or ""): lower(): find("<a%s+href%s*=%s*(['"])axis 2%-admin/logout%1") end }

Example: http-default-accounts (web only) $ sudo nmap -p 80, 443, 8080, 8443 --script=http-default-accounts --script-args=http -default-accounts. categories=web redacted. uchicago. edu PORT STATE SERVICE 80/tcp open http 443/tcp open https 8080/tcp open http-proxy | http-default-accounts: | [Apache Axis 2] at /axis 2 -admin/ |_ admin: axis 2 8443/tcp closed https-alt