Selecting and Implementing An Embedded Database System Presented

Selecting and Implementing An Embedded Database System Presented by Jeff Webb March 2005 Article written by Michael Olson IEEE Software, 2000

Overview ® Key Strategy – Focus on application requirements ® Embedded DB products vary ® Some do less than what you need ® Some will do more ® Choose the tool that most closely matches your needs

Overview ® After choosing OS, HW platform and DB you must design for reliability ® Design for performance up front ® Evaluate performance once the application is built

Evaluate Database Services Available ® High end RDM systems provide: Concurrency ® Transaction processing ® Disaster recovery ® ® Although these features may be needed, enterprise systems are seldom a good choice due to: platform differences ® packaging differences ®

Evaluate Database Services Available ® Several embedded DB systems use the same techniques as enterprise systems but in smaller packages. ® Often, full blown disaster recovery is not needed ® Many embedded databases are configurable allowing inclusion or exclusion of services

Consider Services Required ® Now… consider which services you need. For example: ® Locking? ® Will run faster without locking ® Recovery ® Lack from failures matter? of, or disabling will increase performance

Operating Systems For Embedded systems ® Hundreds of OS for variety of processor hardware ® DB developers have an enormous job porting their software to the variety of platforms ® Few OS dominate the market MS Win CE ® Neutrino ® Vx. Works ® Wind Rivers ®

Databases For Embedded systems ® Classification ® Client Server relational systems ® Client Server OO systems ® Embedded libraries

Classification ® Client ®+ Server relational systems Programmers already know SQL ® - Extra run-time cost of client server communications

Classification ® Client Server OO systems ® Appear to be a good choice however ® Most are designed for Unix systems and their deeply engrained assumptions about memory management and interprocess communications are difficult to port to embedded OS.

Classification ® Embedded ® Created libraries specifically for embedded systems ® Provide simple language API that does not require SQL ® + Faster execution, increased reliability ® - Require developers to master nonstandard programming interfaces

Platform Support ® Consider combination of: ® OS ® Processor ® Storage system ® Exotic processor boards ® Rare combinations can be difficult to fit ® Often embedded OS vendors maintain lists of partner companies whose products run on their systems

Performance Considerations ® High concurrency? ® Size of database ® Multiple control threads? ® Can not rely on standard benchmark measuring systems ® Evaluation of actual performance is a must

Designing the Application ® Design for performance ® Consider: ® Speed ® Predictability & Reliability

Designing the Application Speed Predictability & Reliability ® Data representation ® Access patterns ® Configuration

Designing for Speed Data representation/Access patterns/Configuration ® Most embedded DB tools operate on a fixed set of data types ® Every fetch and store may require translation ® A few systems, mostly library types allow storage in program-native format

Designing for Speed Data representation/ ® Consider Access patterns/Configuration the queries that the application will need to make ® Data should be laid out accordingly ® Can keys be used that will allow related records to be physically stored together? ® B+tree storage typically performs faster than hash table algorithms

Designing for Speed Data representation/Access patterns/ Configuration ® Must understand the chosen systems configuration options ® Memory use for secondary cache ® Write data to disk or store in memory? ® Locking system granularity ® Entire locking system on/off ® Vendors often choose the wrong defaults

Designing the Application ® Design for performance ® Consider: ® Speed ® Predictability & Reliability

Predictability & Reliability ® May need to run with no humans present ® Not easy ® Fanatically check return values and error indicators ® Resource exhaustion ® More common in embedded systems ® Track and release resources yourself

Predictability & Reliability ® Are transactions required so that changes won’t be lost after a crash? ® The recovery system must be callable from the application program on start up?

Performance tuning ® Common causes for poor performance ® Contention – 2 or more threads contending for same data ® Disk-to-memory transfers ® Deadlocks

Contention/Disk-to-memory transfers/ Deadlocks ® When several threads are waiting for the same resources ® Use record level locking if possible ® Use smaller pages if possible to make page level locks more like record level locks ® Touch the Hot data last and hold it for short periods of time

Contention/ Deadlocks ® Disk-to-memory transfers/ latency – mechanical ® Flash RAM ® More memory for buffer cache ® Indexes

Contention/Disk-to-memory transfers/ Deadlocks T 1 waits for a lock on O 2 T 1 T 2 Obj O 2 locked Obj O 1 locked T 2 waits for a lock on O 1

Deadlocks ® Turn off locking if it is not needed and if the application permits ® Break large transactions into several smaller transactions ® Write transactions so that they all acquire the same resources in the same order ® Consider Optimistic Concurrency Control

Price ® Helps make final decision ® Some are available at no cost ® Licensing methods vary Per developer ® Per application using it ® Per deployment platform ® Per user is less common in embedded systems ® ® During development / deployment

Questions