Selecting and Implementing An Embedded Database System Presented
Selecting and Implementing An Embedded Database System Presented by Jeff Webb March 2005 Article written by Michael Olson IEEE Software, 2000
Overview ® Key Strategy – Focus on application requirements ® Embedded DB products vary ® Some do less than what you need ® Some will do more ® Choose the tool that most closely matches your needs
Overview ® After choosing OS, HW platform and DB you must design for reliability ® Design for performance up front ® Evaluate performance once the application is built
Evaluate Database Services Available ® High end RDM systems provide: Concurrency ® Transaction processing ® Disaster recovery ® ® Although these features may be needed, enterprise systems are seldom a good choice due to: platform differences ® packaging differences ®
Evaluate Database Services Available ® Several embedded DB systems use the same techniques as enterprise systems but in smaller packages. ® Often, full blown disaster recovery is not needed ® Many embedded databases are configurable allowing inclusion or exclusion of services
Consider Services Required ® Now… consider which services you need. For example: ® Locking? ® Will run faster without locking ® Recovery ® Lack from failures matter? of, or disabling will increase performance
Operating Systems For Embedded systems ® Hundreds of OS for variety of processor hardware ® DB developers have an enormous job porting their software to the variety of platforms ® Few OS dominate the market MS Win CE ® Neutrino ® Vx. Works ® Wind Rivers ®
Databases For Embedded systems ® Classification ® Client Server relational systems ® Client Server OO systems ® Embedded libraries
Classification ® Client ®+ Server relational systems Programmers already know SQL ® - Extra run-time cost of client server communications
Classification ® Client Server OO systems ® Appear to be a good choice however ® Most are designed for Unix systems and their deeply engrained assumptions about memory management and interprocess communications are difficult to port to embedded OS.
Classification ® Embedded ® Created libraries specifically for embedded systems ® Provide simple language API that does not require SQL ® + Faster execution, increased reliability ® - Require developers to master nonstandard programming interfaces
Platform Support ® Consider combination of: ® OS ® Processor ® Storage system ® Exotic processor boards ® Rare combinations can be difficult to fit ® Often embedded OS vendors maintain lists of partner companies whose products run on their systems
Performance Considerations ® High concurrency? ® Size of database ® Multiple control threads? ® Can not rely on standard benchmark measuring systems ® Evaluation of actual performance is a must
Designing the Application ® Design for performance ® Consider: ® Speed ® Predictability & Reliability
Designing the Application Speed Predictability & Reliability ® Data representation ® Access patterns ® Configuration
Designing for Speed Data representation/Access patterns/Configuration ® Most embedded DB tools operate on a fixed set of data types ® Every fetch and store may require translation ® A few systems, mostly library types allow storage in program-native format
Designing for Speed Data representation/ ® Consider Access patterns/Configuration the queries that the application will need to make ® Data should be laid out accordingly ® Can keys be used that will allow related records to be physically stored together? ® B+tree storage typically performs faster than hash table algorithms
Designing for Speed Data representation/Access patterns/ Configuration ® Must understand the chosen systems configuration options ® Memory use for secondary cache ® Write data to disk or store in memory? ® Locking system granularity ® Entire locking system on/off ® Vendors often choose the wrong defaults
Designing the Application ® Design for performance ® Consider: ® Speed ® Predictability & Reliability
Predictability & Reliability ® May need to run with no humans present ® Not easy ® Fanatically check return values and error indicators ® Resource exhaustion ® More common in embedded systems ® Track and release resources yourself
Predictability & Reliability ® Are transactions required so that changes won’t be lost after a crash? ® The recovery system must be callable from the application program on start up?
Performance tuning ® Common causes for poor performance ® Contention – 2 or more threads contending for same data ® Disk-to-memory transfers ® Deadlocks
Contention/Disk-to-memory transfers/ Deadlocks ® When several threads are waiting for the same resources ® Use record level locking if possible ® Use smaller pages if possible to make page level locks more like record level locks ® Touch the Hot data last and hold it for short periods of time
Contention/ Deadlocks ® Disk-to-memory transfers/ latency – mechanical ® Flash RAM ® More memory for buffer cache ® Indexes
Contention/Disk-to-memory transfers/ Deadlocks T 1 waits for a lock on O 2 T 1 T 2 Obj O 2 locked Obj O 1 locked T 2 waits for a lock on O 1
Deadlocks ® Turn off locking if it is not needed and if the application permits ® Break large transactions into several smaller transactions ® Write transactions so that they all acquire the same resources in the same order ® Consider Optimistic Concurrency Control
Price ® Helps make final decision ® Some are available at no cost ® Licensing methods vary Per developer ® Per application using it ® Per deployment platform ® Per user is less common in embedded systems ® ® During development / deployment
Questions
- Slides: 28