Sci Py Core Next Generation Numeric Background Incompatibilities
Sci. Py Core (Next Generation Numeric) Background ● Incompatibilities ● New Features ● Array Object ● Ufunc Object ● C-API ● Outline of Sci. Py Core ● Roadmap for Release ● Documentation ●
� ● ● ● Background Numarray introduced a lot of great features. Many adopted it. Some needed speed and/or features still only available with Numeric. Result: Split in the community
� Background I'm bothered by the split: ● Encourages some to go elsewhere (Ruby? ) ● Discourages the development of something like Sci. Py: things have to be developed twice. . ● Solution: Add the features introduced by numarray to Numeric and call the result ●
Sci. Py Core ● ● ● The new Numeric (Numeric 24. 0 is the last Numeric). scipy_core is the package name of the replacement (module name is scipy and scipy. <xxxxx>) Core package includes all that Numeric did (linear algebra, FFT's, random numbers, masked arrays, matrices).
Sci. Py Core ● ● Does not rely on Fortran code. Easy to install (need more platforms to test to make sure this is true). A super-enhanced replacement for Numeric. Contains the features of numarray so both camps can come together.
Incompatibilities ● ● ● There is a little bad news --> code will need some adapting to run with the new scipy core distribution. Nearly all of the changes are search-and replace. There is a script called convertcode. py to help.
Importing changes ● ● ● ● Numeric, Mlab, Matrix --> scipy FFT --> scipy. fftpack Linear. Algebra --> scipy. linalg Random. Array --> scipy. stats RNG --> scipy. stats MA --> scipy. ma multiarray --> scipy. base. multiarray umath --> scipy. base. umath
Method Changes ● ● ● ● typecode() --> dtypechar iscontiguous() --> flags['CONTIGUOUS'] byteswapped() --> byteswap() itemsize() --> itemsize spacesaver() gone savespace() gone flat -->. ravel() (mostly)
Other Changes Typecodes ● ● ● ● 'c' --> 'S 1' 'b' --> 'B' '1' --> 'b' 's' --> 'h' 'w' --> 'H' 'u' --> 'I' ● ● typecode= keywords changed to dtype= savespace= keyword replaced with fortran=
New Features There is a lot of great news!!! Core Sci. Py is more powerful than Numeric (and numarray) Core Sci. Py should have little speed hit over Numeric ● ● ● 100% faster in select parts 5 -15% slower in others Needs more testing and benchmarking.
New Features: Arrays ● ● ● Standard arrays inherit from a new bigarray that does not use sequence protocol or buffer protocol. The bigarray can be as large as platform memory allows (no 32 -bit limit on 64 -bit platform) Very new addition. . Needs testing.
New Features: Arrays There are now 21 different data types (10 integers, 3 float, 3 complex, 1 bool, 1 object, 3 flexible).
New Features: Arrays Flexible data types ● ● ● String, Unicode, Void Can have any itemsize. Void can represent any C-structure Data types are true Python types in a hierarchy of types. Data types are all type objects of new Python scalars (21 of them).
New Features: Arrays ● Many methods on arrays instead of just functions. ● ● . tofile(file, sep='', format='') method and fromfile function dump, dumps, __reduce__, __setstate__ view, getfield, setflags reshape, resize, transpose, swapaxes, flatten, ravel, squeeze, take, putmask, repeat, choose, where, sort, argsort, searchsorted, nonzero, compress, diagonal, trace, max, argmax, min, argmin, ptp, clip, conj, sum, cumsum, prod, cumprod, mean, std, all, any, sign, round
New Features: Arrays ● Many new attributes (which reveal and/or modify the array itself) more clearly distinguished from methods ● ● flags : arrays can be misaligned, swapped, and in Fortran order. strides, ndim, data, size, itemsize, base dtype, dtypenum, dtypechar, dtypestr real, imag, flat
New Features: Arrays Fancy Indexing ● ● ● Can use standard slice based indexing Can use Boolean mask Can use integer indexing sequences Can combine slices and integer indexing sequences. All the flexibility of numarray, plus. .
New Features: UFuncs ● ● ● Coercion rules altered for mixed scalar / array operations Needed coercion uses temporary buffers whose size is user controlled. Control on local, module, builtin scope.
New Features: UFuncs ● ● Errors are handled using IEEE floating point status flags. divbyzero, under, over, invalid can be individually set to ignore, warn, raise, and call. Flexibility on local, module, builtin level for handling these errors. Can define a call back function for error handling.
New Features: UFuncs ● ● ● ufunc reduce, accumulate, and reduceat can take place using a different type then the array type (without entire-array being copied). output arrays passed in can be of different data type, then the input. Arbitrary classes with __array__, __array_wrap__, and __array_priority__ can be passed into and returned from ufuncs.
New Features: UFuncs ● UFuncs can be created from Python functions. Ufuncs have several attributes which including doc strings which include their calling syntax. ● nin, nout, nargs, ntypes, identity New ufuncs: (frexp, modf, ldexp, isnan, isfinite, isinf, signbit) ● ●
New Features: C-API ● ● More functionality available from compiled code More macros available for structure access New iterator objects make it easy to handle discontiguous arrays. No magic function lists in C-code. All needed functions are part of data type descriptor.
Overview of Sci. Py Core ● ● ● scipy (contains namespace of scipy. base plus a few select functions) scipy. base (ndarray and ufunc) scipy. lib ●. lapack_lite ●. fftpack_lite ●. random_lite scipy. linalg scipy. fftpack scipy. stats
Overview of Sci. Py Core scipy. distutils Modules not under scipy package structure but included in the distribution: ● ● f 2 py weave
Roadmap to Release ● ● ● Try to released Beta by the end of Sci. Py 2005 conference. Configuration needs the most work (so it can install on every platform desired). Records array and masked arrays need a little work. linalg, fftpack, stats need some grooming. Bugs fixed and optimizations.
Documentation ● ● Guide to Scipy (www. trelgol. com). Can be pre-ordered for discount to get current version. Full version available (hopefully) by the end of October. Released under a MBDR for maximum of 7 years, and a minimum of how long it takes to sell about 10, 000 copies.
Where to get it Current version is on svn. scipy. org under a branch of the scipy_core package. ● ● http: //svn. scipy. org/svn/scipy_core/branches/newcore
- Slides: 26