Splunk Jerry Lee RSTN Taiwan jerry leerstn com

Splunk 初級入門課程 Jerry Lee RSTN , Taiwan jerry. lee@rstn. com. tw (2015年版本 v 6. 3 )

Splunk: 企業 各式資訊機器設備的 營運智慧平台 不用事先定義 資料欄位，不用 客製化 連接器，不用資料庫，不需要事先過濾 資料中心 以外 其他設備資料 客戶使用 資料 Click-stream data Shopping cart data Online transaction data Logfiles 視窗平台 Windows Registry Event logs File system sysinternals Configs Messages Traps Alerts Metrics Scripts Changes Tickets UNIX 平台 Linux/Unix 虛擬化 雲端 Virtual & Cloud 應用系統 Applications 資料庫 Databases Configurations syslog File system ps, iostat, top Hypervisor Guest OS, Apps Cloud Web logs Log 4 J, JMS, JMX. NET events Code and scripts Configurations Audit/query logs Tables Schemas 3 Manufacturing, logistics… CDRs & IPDRs Power consumption RFID data GPS data 網路設備 Networking Configurations syslog SNMP netflow

Splunk �品，四 個主要元件： Search Head, Indexer, Forwarder, Deployment Server 今天入門課程， 安裝軟體的架構 - Log File Search Head Indexing Server Splunk Forwarders 4

Splunk 兩個主要執行程式之一 ： splunkd 全文檢索服務：被�詢、傳回結果、和 將 所有進入的資料 建立索引 Accesses, processes, and indexes incoming data Processes all search requests and returns results Runs a web server on port 8089 by default Speaks SSL by default Splunk helpers run as dependent process(es) of splunkd – Splunk helpers run outside scripts, for example: êScripted inputs êScripted alerts 5

Splunk 兩個主要執行程式之二 ： Splunk Web Python-based web server, based on Cherry. Py framework Provides both search and management web front end for splunkd process Runs on port 8000 by default Sets initial login to user: admin password: changeme 6

Splunk �品的目錄結構 $SPLUNK_HOME bin etc system apps 授權、設定 執行檔 users var lib 安裝套件 splunk 全文檢索處理後 所建立的索引 search <proprietary app> launcher 7

使用 Splunk Web介面之管理員：設定Data Input • Setting up inputs in manager is easy • Useful for learning inputs and their settings • Not typically used for setting production inputs, but can be used to create an example inputs. conf 10

資料輸入 的 類別 – all OS’s 檔案和目錄（Files & directories） Splunk monitors text-based log files 網路輸入（TCP and UDP） - Splunk listens on a specified port for data feeds 指令碼（Scripts） - Splunk runs a script and indexes the output HTTP 事件收集器（HTTP Event Collector） 11

HTTP 事件收集器 （HTTP Event Collector） Supports Dev. Ops and Io. T data analysis needs at scale 1. Standard API and logging libraries send events directly to Splunk 2. Libraries integrated into popular platforms and services Dev. Ops & Developers Io. T Devices & Applications 12 Scales to Millions of Events/Second

指定 資料輸入的 檔案和目錄 add new input edit existing input 13

選擇 輸入的 檔案 或 目錄 位置 => Source • Specify a file or directory for ongoing monitoring • Upload a copy of a file – Useful for testing and development 14

選擇 資料輸入的 指定主機 => host Specify a constant value if all monitored files in an input are from the same host 15

選擇 資料輸入的 來源類型 => sourcetype • Sourcetype is Splunk’s way of identifying the type of data • Default and custom data processing during indexing relies heavily on sourcetype • Also used heavily in searches, reports, dashboards, Apps -- basically the rest of Splunk as well! 16

實際的 資料輸入檔： inputs. conf 的設定 • Each input gets its own stanza - The first line, encased in square brackets [ ], sets the type of input and location - Subsequent lines are “attribute = value” [monitor: ///logs/secure] disabled = false host_segment = 3 sourcetype = linux_secure index = security • See $SPLUNK_HOME/etc/system/README/inputs. c onf. spec for detailed syntax [monitor: ///opt/tradelog. log] disabled = 1 sourcetype = trade_entries host = tradesrv. mycompany. com [udp: //514] connection_host = dns sourcetype = syslog 17

預設可辨識的來源類型，其他可透過下載App、或自行設定 http: //docs. splunk. com/Documentation/Splunk/latest/Data/Listofpretrainedsourcetypes 18

Splunkbase 有800個安裝套件(App)，可免費下載安裝 熱門下載： • Splunk App for Windows • Splunk for Unix and Linux • DB Collect • Splunk for Cisco Firewall • Splunk for F 5 • Splunk for Nagios • Splunk for Web Intelligence • . . http: //apps. splunk. com 22

常見問答，可到 Splunkbase Answer �詢、發問 http: //answers. splunk. com/ 23

跟 Splunk 更多忍者，學習進階技巧 http: //blogs. splunk. com/ http: //wiki. splunk. com/ 24

安裝的 作業系統 和 瀏覽器 需求 Splunk works on Windows, Linux, Solaris, Free. BSD, Mac. OS X, AIX, and HP-UX Firefox 3, 4, and 8; IE 7, 8, and 9; latest Safari and Chrome docs. splunk. com/Documentation/Splunk/latest/Installation/Systemrequire ments 26

Splunk 免費下載（需要先註冊帳號，登入後即可下載） Download Splunk from www. splunk. com/download (login required) Make sure you get the right version for your platform – You might be able to install the wrong version, but it won't run 27

實機上手教材簡報 ＰＤＦ 檔 – 今日教材 （請從 隨身碟 取得） 詳細入門教材檔 – 英文版 ê搜尋入門手冊 êhttp: //docs. splunk. com/Documentation/Splunk/latest/Search. Tutorial/Welcometothe. Se utorial ê資料模型和樞紐分析 教學手冊 êhttp: //docs. splunk. com/Documentation/Splunk/latest/Pivot. Tutorial/Welcometothe. Pivo rial – 中文版 êhttp: //docs. splunk. com/Documentation/Splunk/6. 2. 0/Translated/Traditional. Chinesemanuals 28

Sample Data 範例 範例資料 (Sample Data) – http: //docs. splunk. com/images/Tutorial/tutorialdata. zip – 內含： êApache 1 Log, Apache 2 Log, Apache 3 Log êMail Servers êVendor Sales – 請解開來在 你電腦的某個指定目錄，瀏覽一下～ 線上��表格 (Lookup Table) – http: //docs. splunk. com/images/d/db/Prices. csv. zip – Product ID 對應到 �品名稱 、價格 的 對應表 29

以關鍵字搜尋，可搭配 OR, NOT，可點選 Time. Line 縮小時間 例如： buttercupgames (error OR fail* OR severe) 37

�詢 『 sourcetype=“access_*” 』 40

搜尋範例 搜尋 Buttercup Games 商店的成功購買數 – sourcetype=access_* status=200 action=purchase 搜尋 發生錯誤 的�生記錄 – (error OR fail* OR severe) OR (status=404 OR status=500 OR status=503) 搜尋昨天購買了多少模擬遊戲 – sourcetype=access_* status=200 action=purchase category. Id=simulation 44

搜尋 語言 的 範例 This diagram represents a search, broken into its syntax components Search for this PIPE: Take these events and… PIPE: Take these stats and… sourcetype=access_* status=503 | stats sum(price) as lost_revenue | fieldformat lost_revenue = "$" + tostring(lost_revenue, "commas") COMMAND: Get some stats COMMAND: Format values for the lost_revenue field FUNCTION: Create a string FUNCTION: Get a sum ARGUMENT: Get a sum of the price field ARGUMENT: Format the string from values in the lost_revenue field, insert commas CLAUSE: Call that sum “lost_revenue” 46

搜尋語法處理的過程 （範圍縮小 => 運算 => 呈現） Disk Intermediate results table Final results table sourcetype=syslog ERROR | top user | fields - percent Fetch events from disk that match Summarize into table of top 10 users 47 Remove column showing percentage

將 搜尋結果，透過『進階語法』 統計運算： top 熱門排名 : sourcetype=access_* status=200 action=purchase | top category. Id 48

stats – count by （依欄位分別統計） • The by clause returns a count for each field value of a named field • This example counts the number of events when action=purchase for each product. Id How many of each product was purchased? sourcetype=access_* action=purchase | stats count by product. Id 50

stats – sparkline （�生 分時統計圖） • Used in conjunction with the stats and chart commands What is the purchase trend for each product ID over the last 7 days? • Creates a mini-timeline in a report sourcetype=access* action=purchase | stats sparkline count by product. Id | sort -count - Represents the same time span as the search – in this case “last 7 days” - Not to be confused with timechart, which creates a standalone visualization Note: chart and timechart are covered later in this course 51

利用 eval 進行 �的差異計算 • You can perform mathematical functions against fields with numeric field values How do our prices compare to the competition? sourcetype=access_combined product_name=* | eval difference = price - flowersrus_price | table product_name, price, flowersrus_price, difference • This example compares the flowershop price against the competitor's price - Subtract the value of flowsersrus_price from price - flowersrus_price is another field available via a lookup! 52

圖表指令： chart • This example shows a basic chart • The count function counts the number of events for each http status Are any hosts throwing a lot of errors? sourcetype=access_* | chart count by status 53

時間趨勢圖分析： timechart • This example displays the usage categories over a 1 hour period • Splitting by the usage field, each line represents a unique value of the field • The y-axis represents the count for each field value sourcetype=cisco_w* | timechart count by usage What’s the overall usage trend for the last 24 hours? 54

使用子搜尋（Sub Search ） sourcetype=access_* status=200 action=purchase [search sourcetype=access_* status=200 action=purchase | top limit=1 clientip | table clientip] | stats count, dc(product. Id), values(product. Id) by clientip 57

使用��表格（Lookup Table） 58

確定 自動�� 成功 => 會有 price, product. Name 兩個欄位 sourcetype=access_* 63

選擇 price, product. Name 兩個欄位 64

使用子搜尋（Sub Search ）和 自動�� sourcetype=access_* status=200 action=purchase [search sourcetype=access_* status=200 action=purchase | top limit=1 clientip | table clientip] | stats count AS "Total Purchased", dc(product. Id) AS "Total Products", values(product. Name) AS "Product Names" by clientip | rename clientip AS "VIP Customer 請，另存 為「VIP 客戶」的報告 65

其他 進階搜尋 語法 （一） 1、比較檢視數量和購買數量 （看 => 放進購物車 => 購買） sourcetype=access_* status=200 | chart count AS views count(eval(action=“addtocart”)) AS addtocart count(eval(action=“purchase”)) AS purchases by product. Name | rename product. Name AS “�品名稱 ”, views AS ”瀏覽總量“, addtocart AS “放入購物車總量”, purchases AS ”最終購買總量” 進階語法 （看 => 放進購物車（百分比） => 購買（百分比）） sourcetype=access_* status=200 | stats count AS views count(eval(action="addtocart")) AS addtocart count(eval(action="purchase")) AS purchases by product. Name | eval views. To. Purchase=(purchases/views)*100 | eval cart. To. Purchase=(purchases/addtocart)*100 | table product. Name views addtocart purchases views. To. Purchase cart. To. Purchase | rename product. Name AS “�品名稱”, views AS ”瀏覽總量“, addtocart AS “放入購物車總量”, purchases AS ”最終購買總量 ” 另存為 [�品檢視數以及購買數比較] 報告 66

其他 進階搜尋 語法 （三） 3、購買趨勢 sourcetype=access_* status=200 action=purchase| chart sparkline(count) AS "Purchases Trend" count AS Total by category. Id | rename category. Id AS "Category” 另存為 [購買趨勢] 報告 69

其他 進階搜尋 語法 （四）-1 sourcetype=access_* status=200 | stats count AS views count(eval(action="addtocart")) AS addtocart count(eval(action="purchase")) AS purchases by product. Name | eval views. To. Purchase=(purchases/views)*100 | eval cart. To. Purchase=(purchases/addtocart)*100 | table product. Name views addtocart purchases views. To. Purchase cart. To. Purchase | renam product. Name AS "Product Name" views AS "Views", addtocart as "Adds To Cart", purchases AS "Purchases" 70

製作 儀表版（Dashboard） （一）：建立儀表板 1. 執行下列搜尋 sourcetype=access_* status=200 action=purchase | top category. Id 2. 視覺化選『圓餅圖』 72

地圖功能（Map ） 對應 IP 所在地指令： iplocation [外部IP位址] sourcetype="access_*" | iplocation clientip �生以下欄位 Country 所在國家 City 所在城市 lon 經度 lat 緯度 81

地圖功能（Map ） 統計指令： geostats count by [統計欄位] sourcetype="access_*" | iplocation clientip | geostats count by product. Name 82

資料模型（Data Model）、樞紐分析(Pivot Analysis) 83