Arc IMS 4 Arc IMS Tips Tricks Dan

Arc. IMS 4 Arc. IMS Tips & Tricks Dan Haag November 12, 2002

Overview n Topics to a better Arc. IMS site n n n Performance Scalability Availability Security Manageability Case Studies Arc. IMS 4

Arc. IMS Evolution Arc. IMS 9 Arc. IMS Tracking Server Arc. IMS Arc. Map Server Arc. IMS 4 2002 Arc. IMS Route Server Arc. IMS 3. 1 Arc. IMS 4 2001

Migrating from previous versions n Currently using Arc. IMS 3. 0: n n Wait until you receive Arc. IMS 4. Migrate directly from Arc. IMS 3. 0 to 4, following instructions for 3. 0 to 3. 1 migration. Migrating pdf available at support. esri. com. Currently using Arc. IMS 3. 1: n Arc. IMS 4 Use existing 3. 1 config axls & viewers in 4. 0. No modifications should be required.

System Requirements n Microsoft Windows NT 4. 0, 2000, XP n Sun Solaris, AIX, HP-UX, Linux n Choice of operating system will determine: n Web Server n Java Servlet engine n Application Servers n Java Runtime Environment (JRE) n Detailed requirements at support. esri. com Arc. IMS 4

Installation Tips … General Uninstall Arc. IMS 3. 1/3. 0 completely n Uninstall Servlet Engine & JRE n Install Arc. IMS 4. 0 n Use Site Converter Utility to transfer existing serialization files to new format if desired. n Migrate Viewers & config axl files if going from 3. 0 to 4. 0 n Arc. IMS 4

Installation Tips n Install JRE before anything else n n JRE 1. 3. 1 ships with Arc. IMS Make sure that Web server is up and running Use default port 80 for web server, different one for additional servers n Hit http: //localhost to ensure web server is running, or use web server’s management tool. n n Make sure that the servlet engine is correctly installed and configured with the web server n Can only have ONE servlet engine configured Arc. IMS 4

More Installation Tips n Use test servlets that come with software n Example: Servlet. Exec, test with n n n Make sure virtual directories are in place Adequate system resources for Arc. IMS Spatial Servers Network security issues: Firewalls, DNS issues Use the Arc. IMS Diagnostics Page: n n http: //localhost/servlet/Test. Servlet http: //localhost/servlet/Date. Servlet C: Program FilesArc. GISArc. IMSCommonDiagnosticsArc. IMS_Diagnostics. html Majority of installation problems start and end here!!! Arc. IMS 4

Performance How long does a request take? n Perceived vs. actual performance n n Focus on getting the fastest single user performance by Tuning your data n Tuning your AXLs n Choosing your client applications n Arc. IMS 4

Spatial Server Log Files n Use your spatial server log files to determine performance and “hot spots” n n n n Most popular map services Slowest layers Located in the <Arc. IMS>Serverlog or $AIMSHOME/log directory One log file for each server type: image, feature, query, geocode, extract, arcmap, metadata Provide a record of all requests and responses First place to go if there’s a problem with the response Arc. IMS 4

Turning Log Files on Log files, by default, only log errors n To log all requests and responses: n Open aimsserver. cfg in serveretc directory n Change logfiles=“errors” to logfiles=“on” n Restart monitor and appserver n n More details on log files are available at http: //support. esri. com/ n Arc. IMS 4 Type in the key words “log files”

Tune Your Data n n Shapefiles n n Generalize data by reducing features and/or vertices Make sure your spatial index files (. sbn and. sbx) are in sync with your data Arc. SDE RDBMS tuning n Grid sizes n Arc. IMS 4

Steps to Better AXL Tuning Tune your scale dependencies Where clause vs. Valuemaps 1. 2. • SPATIALQUERY’s searchorder attribute (Arc. SDE only) 3. • 4. Arc. IMS 4 Use where clauses when you can reduce the number of features found. Use “attributefirst” if the query will reduce the amount of features queried more than the filter. Use Strings & Integers in Valuemaps

Steps to Better AXL Tuning 5. Use Featurelimits for large layers • 6. 7. Minimize the use of antialiasing and transparency Minimize multilayered symbols • Arc. IMS 4 To avoid having large queries bog down your system Cased highways, etc.

Steps to Better AXL Tuning Choose the right output image (GIF, JPG, PNG-8 bit, PNG-24 bit) 8. n n JPG, PNG-24 bit for > 256 Colors GIF, PNG-8 bit for < 256 Colors Use either pyramided raster types (Mr. SID, Arc. SDE) or uncompressed imagery (TIFF) Avoid projection on the fly 9. 10. 1. Arc. IMS 4 Store data in geographic coordinates (WGS 84) if possible.

Tune Your Viewers n Choose the client that’s right for your applications: Thick—Java, Active. X, Arc. Explorer, Arc. GIS n Thin—Java. Script, DHTML n Server Side—HTML from Cold. Fusion, JSP, ASP… n Arc. IMS 4

Tune Your Viewers n Bandwidth n n n 28. 8 k connection use server side or thin DSL or Intranet can use thick Functionality n n Arc. IMS 4 Simple functionality (pan, zoom, id)—thin Sophisticated geoprocessing—thick

Tune Your Viewers Default viewers are meant to be generic. n Modify them so that they are optimized for your data and application. n If you need major customization, use the server side connectors (Java, Active. X, or Cold. Fusion). n Arc. IMS 4

Scalability n How can you maintain your single user performance across multiple users? Tune your Arc. IMS instances n Hardware (capacity planning) n Arc. IMS 4

Understanding Instances Arc. IMS 4

Understanding Instances Arc. IMS 4

How Many Arc. IMS Instances Do I Need? n Answer: Approximately one for each simultaneous request n Simultaneous users are not the same as simultaneous requests! n Can estimate simultaneous requests from average requests per day n Example: 300, 000 avg. req/day Arc. IMS 4

Average Request/Second Estimate n From avg. req/day find avg. req/s n Might use 80/20 rule n 80% of Web traffic occurs 20% of the time n For 300, 000 avg. req/day, this is 14 avg. req/s. Arc. IMS 4

Maximum Request/Second Estimate n Poisson distribution can be used to estimate the maximum req/s. • P(x) is the probability of getting x simultaneous requests with an avg. req/s of l. Arc. IMS 4

Some Examples Arc. IMS 4

Some Examples Arc. IMS 4 Reqs/day Avg. Reqs/s Max. Reqs/s 10, 000 1 5– 6 50, 000 3 7– 8 100, 000 5 13– 14 215, 000 10 20– 21 300, 000 14 26– 27 430, 000 20 34– 35 500, 000 24 39– 40 648, 000 30 47 864, 000 40 59 1, 000 47 67– 68 1, 080, 000 50 71 1, 500, 00 70 94 2, 000 93 120 2, 160, 000 128 n For 300, 000 avg. req/day, you will at most get 26– 27 simultaneous requests.

What Does All This Mean? For 300, 000 -> expect 27 requests/second n Need to scale your Arc. IMS site to handle 27 simultaneous requests: n n Arc. IMS 4 Set the number of instances to 27 if you don’t want requests to wait in the queue

How Many Arc. IMS Machines Do I Need? n Rule of thumb 4– 8 instances of Image. Server per CPU n 5– 15 instances of Feature. Server per CPU n No more than 10 instances per spatial server n n Some variables affect this Arc. IMS 4

Scalability Factors n Arc. IMS 4 Shapefiles n Faster setup n Faster single hit performance n More CPU usage on Map. Server Arc. SDE n Better maintainability n Better scalability n Offset CPU to DB server Sha n pe Arc. SDE vs. Shapefiles Retrieval Time n DE Arc. S Number of Users

Scalability Factors n Vector vs. Raster n n Raster decompression (Mr. SID, JPG, etc. ) requires more CPU on spatial server. Feature. Server vs. Image. Server Feature. Server only retrieves features and streams them (compressed or decompressed). n Image. Server retrieves and renders features and therefore takes more CPU. n Arc. IMS 4

Other Scalability Factors n Serverside business logic Cold. Fusion/JSP/ASP n Extra processing on Web server machines n Arc. IMS 4

Other Scalability Factors n Make sure you have enough bandwidth n Image. Server n n Average Image Size * req/s = Mbits/s Example 1: n n Example 2: n n 100 k JPG * 50 = 5 MBytes/s = 40 Mbits/s = Large T 3! 10 k PNG * 10 =. 1 MBytes/s =. 8 Mbits/s = T 1 Feature. Server n Depends n n Arc. IMS 4 Compression ~10– 20 times Number of features transferred

Virtual Servers n What are they? n n Groups of instances across machines Why do we need them? n Arc. IMS 4 To be able to assign services to different spatial servers/machines

Virtual Servers n Recommendation n n Use only the default virtual server Except if n Need to prioritize services n e. g. , Services A and B are more important and need to run on the 1. 5 GHz machine. Services C and D are less important and can run on the 133 MHz machine. Arc. IMS 4

Availability How much time is your site available for use? n Components will fail. n n Often measured in percentage uptime. Arc. IMS 4 95% 99. 99% 99. 999% Days/Yr 18. 25 3. 65 1. 825 0. 365 0. 00365 Hr/Yr 438 87. 6 43. 8 8. 76 0. 876 0. 0876 Min/Yr 26280 5256 2628 525. 6 52. 56 5. 256

How to Maximize Availability n Machine availability n n Use UPSs on all machines Redundant power supplies Everything hot swappable Data availability n Use a RAID configuration n n Arc. IMS 4 Level 0—striping Level 1—mirroring Level 5—parity Level 0+1—mirrored and striped

Application Availability n Web servers n n Multiple Web servers with a load balancer Arc. IMS availability n n Distribute components on different machines Multiple spatial servers n n n Multiple Arc. IMS App. Servers n Arc. IMS 4 Always run multiple spatial servers Distribute spatial servers on multiple machines Consider running multiple Arc. IMS App. Servers

Network Availability Multiple Internet Connections n Backup Firewalls n Backup Routers n Backup Switches n Arc. IMS 4

Staging & Production n Never develop and test on your production system. n Development n n Staging n n Develop and test new applications Test new applications under production conditions Production n Arc. IMS 4 Move to production when ready

Monitor, Monitor Even if Web site is designed for 99. 999% availability, things will still fail. n Monitoring can be done using n Simple scripts: perl, awk, bat n Commercial tools (SNMP, etc. ) n Alerts can easily be sent to a pager n Arc. IMS 4

Security How secure is your site? n How long would it take for someone to breakin? n What would be the costs related to a break-in? n Arc. IMS 4

Firewalls A firewall allows control of access to a machine from the network. n It filters out packets of data based on a set of rules. n n Arc. IMS 4 For instance: allow all access to port 80 (http) and disallow all other ports on the system (ftp, telnet, etc. ).

Recommend: DMZ Network Internet DMZ Web Server Arc. IMS Server Intranet Arc. IMS 4

Most Popular n Web Server Outside Firewall Port 5300 Open n File sharing from Intranet to Web server for image output n Intranet Internet Web Server Arc. IMS 4 App. Server Spatial Server Data

Not Recommended n Connection between App. Server and spatial server persistent Http 5353 Web. Server + App. Server Arc. IMS 4 Spatial Servers

Service Authentication n Access to Arc. IMS services can be restricted Only people with proper credentials can access services n If information for accessing service is not correct, request is not sent to the Arc. IMS Application Server, and an error message is returned to client n Process is called authentication n Arc. IMS 4

Limiting Access By user name and password n By IP address n By setting an expiration date n By limiting the number of times a user can access the Map. Service n By limiting request elements such as GET_IMAGE or GET_FEATURES n Arc. IMS 4

Access Control List (ACL) n Text file based n n n JDBC-based ACL n n XML based using XML elements File is loaded in memory Cumbersome to manage with many users Must restart servlet engine after every change Store permissions in a relational database accessed through a JDBC driver Easier to manage large number of users Changes can be made without restarting the servlet engine If ACL is used, services are not accessible unless listed Arc. IMS 4

Arc. IMS Authentication n Use Digest Authentication n n Not Basic Use the RDBMS Acl over the file-based Acl Can dynamically add/remove/update users n Single place to store privileges n Arc. IMS 4

Manageability How manageable is your system? n Use fewer machines! n Use standard paths on all machines. n Come up with a standard methodology for installations, patches, etc. n Document all installs and changes made. n Arc. IMS 4

Case Studies

One Machine Scenario n Internet Capacity 30, 000 maps/day (6 max. req/s) n 95% availability (18 days/yr) n n Configuration n n Arc. IMS 4 1 machine running everything (Web server, Arc. IMS, shapefiles) Can support about 4– 6 simultaneous requests

Two Machine Scenario Internet n Capacity n n Web Server App. Server n Configuration n n Spatial Server Arc. IMS 4 n 50, 000 maps/day (8 max. req/s) 95% availability (18 days/yr) 1 machine for Web server and App. Server 1 machine for spatial server and shapefiles Can support about 6– 8 simultaneous requests

Three Machine + DB Server Scenario Internet n Capacity n n Web Server App. Server n Configuration n Spatial Server n n Arc. SDE n Server Arc. IMS 4 150, 000 maps/day (16 max. req/s) 99% availability (3. 6 days/yr) 1 machine for Web server and App. Server 2 machines for spatial server 1 Arc. SDE server Can support about 16 simultaneous requests

Five Machine + DB Server Scenario n Internet Requirements n Web Servers n App. Server Spatial Server n Configuration n Spatial Servers n n n Arc. SDE Server Arc. IMS 4 n 250, 000 maps/day (24 max. req/s) 99. 5% availability (43 hrs/yr) 2 machines for Web server and App. Server 1 machine as App. Server/spatial server 2 machines for spatial server 1 Arc. SDE server Can support about 24

A Large Size Mapping System Web Server Arc. IMS 4 App Servers Spatial Servers Arc. SDE Server

ESRI Geography Network Node Arc. IMS 4

ESRI Geography Network Node Global Load Balancer Redlands Arc. IMS 4 Off-Site Data Center

Hardware Configuration n Data servers n Sun Ultra Enterprise 6500 s n n 12– 360 MHz Processors 8 GB of Random Access Memory 1 TB of Disk Storage Web servers n U 220 Rs, E 250 s n n Arc. IMS 4 1 GB– 2 GB RAM 2– 450 MHz Processors

Hardware Configuration n App. Servers/Spatial Servers n E 450 s, U 420 Rs n n n 4 GB RAM 4– 450 MHz Processors Spatial Servers n E 250 s, U 220 Rs, U 420 Rs n n Arc. IMS 4 2 GB RAM 2– 450 MHz Processors

Site Stats Machines: 44 n Maps generated per day: 1 M+ n Peak per hours: 100 k+ n Average map generation time: 1. 6 s n Map services: 130 n Availability requirements: 99. 5% n n Estimated capacity per day: 5 M+ Arc. IMS 4

Need more information…. ESRI’s Web Site: www. esri. com Arc. IMS Resources: support. esri. com Dan Haag Phone: 636 -949 -6620 x 8523 Email: dhaag@esri. com Arc. IMS 4