class Amalgalite::Database
The encapsulation of a connection to an SQLite3 database.
Example opening and possibly creating a new database
db = Amalgalite::Database.new( "mydb.db" ) db.execute( "SELECT * FROM table" ) do |row| puts row end db.close
Open a database read only:
db = Amalgalite::Database.new( "mydb.db", "r" )
Open an in-memory database:
db = Amalgalite::MemoryDatabase.new
Constants
- VALID_MODES
list of valid modes for opening an Amalgalite::Database
Attributes
A list of the user defined aggregates
the low level Amalgalite::SQLite3::Database
A list of the user defined functions
An object that follows the ProfileTap protocol, or nil. By default this is nil
An object that follows the TraceTap protocol, or nil. By default this is nil
An object that follows the TypeMap protocol, or nil. By default this is an instances of TypeMaps::DefaultMap
Public Class Methods
Create a new Amalgalite database
The first parameter is the filename of the sqlite database. Specifying “:memory:” as the filename creates an in-memory database.
The second parameter is the standard file modes of how to open a file.
The modes are:
* r - Read-only * r+ - Read/write, an error is thrown if the database does not already exist * w+ - Read/write, create a new database if it doesn't exist
w+
is the default as this is how most databases will want to
be utilized.
opts is a hash of available options for the database:
* :utf16 option to set the database to a utf16 encoding if creating a database.
By default, databases are created with an encoding of utf8. Setting this to true and opening an already existing database has no effect.
NOTE Currently :utf16 is not supported by Amalgalite, it is planned for a later release
# File lib/amalgalite/database.rb, line 139 def initialize( filename, mode = "w+", opts = {}) @open = false @profile_tap = nil @trace_tap = nil @type_map = ::Amalgalite::TypeMaps::DefaultMap.new @functions = Hash.new @aggregates = Hash.new @utf16 = false unless VALID_MODES.keys.include?( mode ) raise InvalidModeError, "#{mode} is invalid, must be one of #{VALID_MODES.keys.join(', ')}" end if not File.exist?( filename ) and opts[:utf16] then raise NotImplementedError, "Currently Amalgalite has not implemented utf16 support" else @api = Amalgalite::SQLite3::Database.open( filename, VALID_MODES[mode] ) end @open = true end
Public Instance Methods
Is the database in autocommit mode or not
# File lib/amalgalite/database.rb, line 180 def autocommit? @api.autocommit? end
clear all the current taps
# File lib/amalgalite/database.rb, line 340 def clear_taps! self.trace_tap = nil self.profile_tap = nil end
Close the database
# File lib/amalgalite/database.rb, line 170 def close if open? then @api.close @open = false end end
Commit a transaction
# File lib/amalgalite/database.rb, line 672 def commit execute( "COMMIT TRANSACTION" ) if in_transaction? end
Define an SQL aggregate function, these are functions like max(), min(), avg(), etc. SQL functions that would be used when a GROUP BY clause is in effect. See also ::Amalgalite::Aggregate.
A new instance of MyAggregateClass is created for each instance that the SQL aggregate is mentioned in SQL.
# File lib/amalgalite/database.rb, line 769 def define_aggregate( name, klass ) db_aggregate = klass a = klass.new raise AggregateError, "Use only mandatory or arbitrary parameters in an SQL Aggregate, not both" if a.arity < -1 raise AggregateError, "Aggregate implementation name '#{a.name}' does not match defined name '#{name}'" if a.name != name @api.define_aggregate( name, a.arity, klass ) @aggregates[a.signature] = db_aggregate nil end
Register a busy handler for this database connection, the handler MUST
follow the to_proc
protocol indicating that is will
+respond_to?(:call)+. This is intrinsic to lambdas and blocks so those
will work automatically.
This exposes the sqlite busy handler api to ruby.
* sqlite.org/c3ref/busy_handler.html
The busy handler's _call(N)_ method may be invoked whenever an attempt
is made to open a database table that another thread or process has locked.
N
will be the number of times the _call(N)_ method has been
invoked during this locking event.
The handler may or maynot be called based upon what SQLite determins.
If the handler returns nil or false then no more busy handler calls will be made in this lock event and you are probably going to see an SQLite::Error in your immediately future in another process or in another piece of code.
If the handler returns non-nil or non-false then another attempt will be made to obtain the lock, lather, rinse, repeat.
If an Exception happens in a busy handler, it will be the same as if the busy handler had returned nil or false. The exception itself will not be propogated further.
# File lib/amalgalite/database.rb, line 855 def define_busy_handler( callable = nil, &block ) handler = ( callable || block ).to_proc a = handler.arity raise BusyHandlerError, "A busy handler expects 1 and only 1 argument, not #{a}" if a != 1 @api.busy_handler( handler ) end
register a callback to be exposed as an SQL function. There are multiple ways to register this function:
1. db.define_function( “name” ) { |a| … } * pass
define_function
a name and a block. * The SQL
function name taking arity parameters will be registered,
where arity is the arity of the block. * The return value
of the block is the return value of the registred SQL function 2.
db.define_function( “name”, callable ) * pass function
a
name and something that responds_to?( :to_proc )
*
The SQL function name is registered taking arity
parameters is registered where arity is the arity of
callable.to_proc.call
* The return value of the
callable.to_proc.call
is the return value of the SQL function
See also ::Amalgalite::Function
# File lib/amalgalite/database.rb, line 710 def define_function( name, callable = nil, &block ) p = ( callable || block ).to_proc raise FunctionError, "Use only mandatory or arbitrary parameters in an SQL Function, not both" if p.arity < -1 db_function = ::Amalgalite::SQLite3::Database::Function.new( name, p ) @api.define_function( db_function.name, db_function ) @functions[db_function.signature] = db_function nil end
Register a progress handler for this database connection, the handler MUST
follow the to_proc
protocol indicating that is will
+respond_to?(:call)+. This is intrinsic to lambdas and blocks so those
will work automatically.
This exposes the sqlite progress handler api to ruby.
* sqlite.org/c3ref/progress_handler.html
The progress handler's _call()_ method may be invoked ever N SQLite op
codes. If the progress handler returns anything that can evaluate to
true
then current running sqlite statement is terminated at
the earliest oppportunity.
You can use this to be notified that a thread is still processingn a request.
# File lib/amalgalite/database.rb, line 910 def define_progress_handler( op_code_count = 25, callable = nil, &block ) handler = ( callable || block ).to_proc a = handler.arity raise ProgressHandlerError, "A progress handler expects 0 arguments, not #{a}" if a != 0 @api.progress_handler( op_code_count, handler ) end
return the encoding of the database
# File lib/amalgalite/database.rb, line 220 def encoding @encoding ||= pragma( "encoding" ).first['encoding'] end
SQL escape the input string
# File lib/amalgalite/database.rb, line 194 def escape( s ) Amalgalite::SQLite3.escape( s ) end
helper for an exclusive transaction
# File lib/amalgalite/database.rb, line 594 def exclusive_transaction( &block ) transaction( TransactionBehavior::EXCLUSIVE, &block ) end
Execute a single SQL statement.
If called with a block and there are result rows, then they are iteratively yielded to the block.
If no block is passed, then all the results are returned as an arrayfields instance. This is an array with field name access.
If no block is passed, and there are no results, then an empty Array is returned.
On an error an exception is thrown
This is just a wrapper around the preparation of an Amalgalite Statement and iterating over the results.
# File lib/amalgalite/database.rb, line 294 def execute( sql, *bind_params ) stmt = prepare( sql ) stmt.bind( *bind_params ) if block_given? then stmt.each { |row| yield row } else return stmt.all_rows end ensure stmt.close if stmt end
Execute a batch of statements, this will execute all the sql in the given string until no more sql can be found in the string. It will bind the same parameters to each statement. All data that would be returned from all of the statements is thrown away.
Returns¶ ↑
All statements to be executed in the batch must be terminated with a ';' Returns the number of statements executed
# File lib/amalgalite/database.rb, line 316 def execute_batch( sql, *bind_params) count = 0 while sql prepare( sql ) do |stmt| stmt.execute( *bind_params ) sql = stmt.remaining_sql sql = nil unless (sql.index(";") and Amalgalite::SQLite3.complete?( sql )) end count += 1 end return count end
Execute a sql statment, and only return the first row of results. This is a shorthand method when you only want a single row of results from a query. If there is no result, then return an empty array
It is in all other was, exactly like execute()
# File lib/amalgalite/database.rb, line 352 def first_row_from( sql, *bind_params ) stmt = prepare( sql ) stmt.bind( *bind_params) row = stmt.next_row || [] stmt.close return row end
Execute an sql statement, and return only the first column of the first row. If there is no result, return nil.
It is in all other ways, exactly like first_row_from()
# File lib/amalgalite/database.rb, line 366 def first_value_from( sql, *bind_params ) return first_row_from( sql, *bind_params).first end
helper for an immediate transaction
# File lib/amalgalite/database.rb, line 589 def immediate_transaction( &block ) transaction( TransactionBehavior::IMMEDIATE, &block ) end
Execute a batch of statements via sqlite3_exec. This does the same as #execute_batch, but doesn't update the statement statistics.
# File lib/amalgalite/database.rb, line 333 def import(sql) @api.execute_batch(sql) end
#import_csv_to_table() takes 2 required arguments, and a hash of options. The first argument is the path to a CSV, the second is the table in which to load the data. The options has is a subset of those used by CSV
* :col_sep - the string placed between each field. Default is “,” * :row_sep - the String appended to the end of each row. Default is :auto * :quote_char - The character used to quote fields. Default '“' * :headers - set to true or :first_row if there are headers in this CSV. Default is false. This may also be an Array. If that is the case then the array is used as the fields in the CSV and the fields in the table in which to insert. If this is set to an Array, it is assumed that all rows in the csv will be inserted.
# File lib/amalgalite/database.rb, line 978 def import_csv_to_table( csv_path, table_name, options = {} ) importer = CSVTableImporter.new( csv_path, self, table_name, options ) importer.run end
return whether or not the database is currently in a transaction or not
# File lib/amalgalite/database.rb, line 227 def in_transaction? not @api.autocommit? end
Cause another thread with a handle on this database to be interrupted and return at the earliest opportunity as interrupted. It is not safe to call this method if the database might be closed before interrupt! returns.
# File lib/amalgalite/database.rb, line 880 def interrupt! @api.interrupt! end
Return the rowid of the last inserted row
# File lib/amalgalite/database.rb, line 187 def last_insert_rowid @api.last_insert_rowid end
Is the database open or not
# File lib/amalgalite/database.rb, line 163 def open? @open end
Prepare a statement for execution
If called with a block, the statement is yielded to the block and the statement is closed when the block is done.
db.prepare( "SELECT * FROM table WHERE c = ?" ) do |stmt| list_of_c_values.each do |c| stmt.execute( c ) do |row| puts "when c = #{c} : #{row.inspect}" end end end
Or without a block:
stmt = db.prepare( "INSERT INTO t1(x, y, z) VALUES ( :
# File lib/amalgalite/database.rb, line 264 def prepare( sql ) stmt = Amalgalite::Statement.new( self, sql ) if block_given? then begin yield stmt ensure stmt.close stmt = nil end end return stmt end
Register a profile tap.
Registering a profile tap means that the obj
registered will
have its profile
method called with an Integer and a String
parameter every time a profile event happens. The Integer is the number of
nanoseconds it took for the String (SQL) to execute in wall-clock time.
That is, every time a profile event happens in SQLite the following is invoked:
obj.profile( str, int )
For instance:
db.profile_tap = Amalgalite::ProfileTap.new( logger, 'debug' )
This will register an instance of ProfileTap,
which wraps an logger object. On each profile
event the Amalgalite::ProfileTap#profile
method will be called which in turn will call
<tt>logger.debug<tt> with a formatted string containing the
String and Integer from the profile event.
db.profile_tap = nil
This will unregister the profile tap
# File lib/amalgalite/database.rb, line 458 def profile_tap=( tap_obj ) # unregister any previous profile tap unless @profile_tap.nil? @profile_tap.profile( 'unregistered as profile tap', 0.0 ) @profile_tap = nil end return @profile_tap if tap_obj.nil? if tap_obj.respond_to?( 'profile' ) then @profile_tap = tap_obj else raise Amalgalite::Error, "#{tap_obj.class.name} cannot be used to tap. It has no 'profile' method" end @api.register_profile_tap( @profile_tap ) @profile_tap.profile( 'registered as profile tap', 0.0 ) end
Surround the give string with single-quotes and escape any single-quotes in the string
# File lib/amalgalite/database.rb, line 201 def quote( s ) Amalgalite::SQLite3.quote( s ) end
Release a savepoint. This is similar to a commit but only for savepoints. All savepoints up the savepoint stack and include the name savepoint being released are 'committed' to the transaction. There are several ways of thinking about release and they are all detailed in the sqlite documentation: sqlite.org/lang_savepoint.html
# File lib/amalgalite/database.rb, line 655 def release( point_name ) execute( "RELEASE SAVEPOINT #{point_name}" ) if in_transaction? end
By default once the schema is obtained, it is cached. This is here to force the schema to be reloaded.
# File lib/amalgalite/database.rb, line 516 def reload_schema!( dbname = "main" ) @schema = nil schema( dbname ) end
Remove an aggregate from use in the database. Since the same aggregate may be refistered more than once with different arity, you may specify the arity, or the aggregate class, or nil. If nil is used for the arity then Amalgalite does its best to remove all aggregates of the given name
# File lib/amalgalite/database.rb, line 790 def remove_aggregate( name, klass_or_arity = nil ) klass = nil case klass_or_arity when Integer arity = klass_or_arity when NilClass arity = nil else klass = klass_or_arity arity = klass.new.arity end to_remove = [] if arity then signature = ::Amalgalite::SQLite3::Database::Function.signature( name, arity ) db_aggregate = @aggregates[ signature ] raise AggregateError, "db aggregate '#{name}' with arity #{arity} does not appear to be defined" unless db_aggregate to_remove << db_aggregate else possibles = @aggregates.values.select { |a| a.new.name == name } raise AggregateError, "no db aggregate '#{name}' appears to be defined" if possibles.empty? to_remove = possibles end to_remove.each do |db_agg| i = db_agg.new @api.remove_aggregate( i.name, i.arity, db_agg) @aggregates.delete( i.signature ) end end
Remove the busy handler for this database connection.
# File lib/amalgalite/database.rb, line 868 def remove_busy_handler @api.busy_handler( nil ) end
Remove a function from use in the database. Since the same function may be registered more than once with different arity, you may specify the arity, or the function object, or nil. If nil is used for the arity, then Amalgalite does its best to remove all functions of given name.
# File lib/amalgalite/database.rb, line 732 def remove_function( name, callable_or_arity = nil ) arity = nil if callable_or_arity.respond_to?( :to_proc ) then arity = callable_or_arity.to_proc.arity elsif callable_or_arity.respond_to?( :to_int ) then arity = callable_or_arity.to_int end to_remove = [] if arity then signature = ::Amalgalite::SQLite3::Database::Function.signature( name, arity ) db_function = @functions[ signature ] raise FunctionError, "db function '#{name}' with arity #{arity} does not appear to be defined" unless db_function to_remove << db_function else possibles = @functions.values.select { |f| f.name == name } raise FunctionError, "no db function '#{name}' appears to be defined" if possibles.empty? to_remove = possibles end to_remove.each do |db_func| @api.remove_function( db_func.name, db_func ) @functions.delete( db_func.signature ) end end
Remove the progress handler for this database connection.
# File lib/amalgalite/database.rb, line 923 def remove_progress_handler @api.progress_handler( nil, nil ) end
#replicate_to() takes a single argument, either a String or an Amalgalite::Database. It returns the replicated database object. If given a String, it will truncate that database if it already exists.
Replicate the current database to another location, this can be used for a number of purposes:
* load an sqlite database from disk into memory * snaphost an in memory db and save it to disk * backup on sqlite database to another location
# File lib/amalgalite/database.rb, line 944 def replicate_to( location ) to_db = nil case location when String to_db = Amalgalite::Database.new( location ) when Amalgalite::Database to_db = location else raise ArgumentError, "replicate_to( #{location} ) must be a String or a Database" end @api.replicate_to( to_db.api ) return to_db end
Rollback a transaction
# File lib/amalgalite/database.rb, line 679 def rollback execute( "ROLLBACK TRANSACTION" ) if in_transaction? end
Rollback to a savepoint. The transaction is not cancelled, the transaction is restarted.
# File lib/amalgalite/database.rb, line 665 def rollback_to( point_name ) execute( "ROLLBACK TO SAVEPOINT #{point_name}" ) end
return how many rows changed in the last insert, update or delete statement.
# File lib/amalgalite/database.rb, line 234 def row_changes @api.row_changes end
Much of the following documentation is para-phrased from sqlite.org/lang_savepoint.html
Savepoints are a method of creating transactions, similar to transaction except that they may be nested.
* Every savepoint must have a name, to_s
is called on the
method argument * A savepoint does not need to be initialized inside a
transaction. If it is not inside a transaction it
behaves exactly as if a DEFERRED transaction had been started. * If a block
is passed to saveponit then when the block exists, it is
guaranteed that either a 'RELEASE' or 'ROLLBACK TO name'
has been executed. * If any exception happens during the savepoint
transaction, then a 'ROLLOBACK TO' is issued when the block closes.
* If no exception happens during the transaction then a 'RELEASE
name' is issued upon leaving the block
If no block is passed in then you are on your own.
# File lib/amalgalite/database.rb, line 625 def savepoint( name ) point_name = name.to_s.strip raise Amalgalite::Error, "Invalid savepoint name '#{name}'" unless point_name and point_name.length > 1 execute( "SAVEPOINT #{point_name};") if block_given? then begin return ( yield self ) ensure if $! then rollback_to( point_name ) raise $! else release( point_name ) end end else return in_transaction? end end
return how many rows have changed since this connection to the database was opened.
# File lib/amalgalite/database.rb, line 242 def total_changes @api.total_changes end
Register a trace tap.
Registering a trace tap measn that the obj
registered will
have its trace
method called with a string parameter at
various times. If the object doesn't respond to the trace
method then write
will be called.
For instance:
db.trace_tap = Amalgalite::TraceTap.new( logger, 'debug' )
This will register an instance of TraceTap,
which wraps an logger object. On each trace
event the Amalgalite::TraceTap#trace method
will be called, which in turn will call the logger.debug
method
db.trace_tap = $stderr
This will register the $stderr
io stream as a trace tap.
Every time a trace
event happens then $stderr.write( msg
)
will be called.
db.trace_tap = nil
This will unregistere the trace tap
# File lib/amalgalite/database.rb, line 399 def trace_tap=( tap_obj ) # unregister any previous trace tap # unless @trace_tap.nil? @trace_tap.trace( 'unregistered as trace tap' ) @trace_tap = nil end return @trace_tap if tap_obj.nil? # wrap the tap if we need to # if tap_obj.respond_to?( 'trace' ) then @trace_tap = tap_obj elsif tap_obj.respond_to?( 'write' ) then @trace_tap = Amalgalite::TraceTap.new( tap_obj, 'write' ) else raise Amalgalite::Error, "#{tap_obj.class.name} cannot be used to tap. It has no 'write' or 'trace' method. Look at wrapping it in a Tap instances." end # and do the low level registration # @api.register_trace_tap( @trace_tap ) @trace_tap.trace( 'registered as trace tap' ) end
Begin a transaction. The valid transaction types are:
DEFERRED:: no read or write locks are created until the first statement is executed that requries a read or a write IMMEDIATE:: a readlock is obtained immediately so that no other process can write to the database EXCLUSIVE:: a read+write lock is obtained, no other proces can read or write to the database
As a convenience, these are constants available in the Database::TransactionBehavior class.
Amalgalite Transactions are database level transactions, just as SQLite's are.
If a block is passed in, then when the block exits, it is guaranteed that either 'COMMIT' or 'ROLLBACK' has been executed.
If any exception happens during the transaction that is caught by Amalgalite, then a 'ROLLBACK' is issued when the block closes.
If no exception happens during the transaction then a 'COMMIT' is issued upon leaving the block.
If no block is passed in then you are on your own.
Nesting a transaaction via the transaction method are no-ops. If you call transaction within a transaction, no new transaction is started, the current one is just continued.
True nexted transactions are available through the savepoint method.
# File lib/amalgalite/database.rb, line 562 def transaction( mode = TransactionBehavior::DEFERRED, &block ) raise Amalgalite::Error, "Invalid transaction behavior mode #{mode}" unless TransactionBehavior.valid?( mode ) # if already in a transaction, no need to start a new one. if not in_transaction? then execute( "BEGIN #{mode} TRANSACTION" ) end if block_given? then begin previous_exception = $! return ( yield self ) ensure if $! and ($! != previous_exception) then rollback raise $! else commit end end else return in_transaction? end end
Assign your own TypeMap instance to do type
conversions. The value assigned here must respond to
bind_type_of
and result_value_of
methods. See
the TypeMap class for more details.
# File lib/amalgalite/database.rb, line 485 def type_map=( type_map_obj ) %w[ bind_type_of result_value_of ].each do |method| unless type_map_obj.respond_to?( method ) raise Amalgalite::Error, "#{type_map_obj.class.name} cannot be used to do type mapping. It does not respond to '#{method}'" end end @type_map = type_map_obj end
Is the database utf16 or not? A database is utf16 if the encoding is not UTF-8. Database can only be UTF-8 or UTF-16, and the default is UTF-8
# File lib/amalgalite/database.rb, line 209 def utf16? return @utf16 #if @utf16.nil? # @utf16 = (encoding != "UTF-8") #end #return @utf16 end