database¶
database_interface.py¶
A module to interact with the JWQL postgresql database jwqldb
The load_connection()
function within this module allows the user
to connect to the jwqldb
database via the session
, base
,
and engine
objects (described below). The classes within serve as
ORMs (Object-relational mappings) that define the individual tables of
the relational database.
The engine
object serves as the low-level database API and perhaps
most importantly contains dialects which allows the sqlalchemy
module to communicate with the database.
The base
object serves as a base class for class definitions. It
produces Table
objects and constructs ORMs.
The session
object manages operations on ORM-mapped objects, as
construced by the base. These operations include querying, for
example.
Use¶
Executing the module on the command line will build the database tables defined within:
python database_interface.pyUsers wishing to interact with the existing database may do so by importing various connection objects and database tables, for example:
from jwql.database.database_interface import Anomaly from jwql.database.database_interface import session results = session.query(Anomaly).all()
Dependencies¶
The user must have a configuration file named
config.json
placed in thejwql
directory.
- class jwql.database.database_interface.CentralStore(**kwargs)[source]¶
ORM for the central storage area filesystem monitor table
- area¶
- available¶
- date¶
- id¶
- size¶
- used¶
- jwql.database.database_interface.FGSAnomaly¶
alias of
fgs_anomaly
- jwql.database.database_interface.FGSBadPixelQueryHistory¶
alias of
fgs_bad_pixel_query_history
- jwql.database.database_interface.FGSBadPixelStats¶
alias of
fgs_bad_pixel_stats
- jwql.database.database_interface.FGSCosmicRayQueryHistory¶
alias of
fgs_cosmic_ray_query_history
- jwql.database.database_interface.FGSCosmicRayStats¶
alias of
fgs_cosmic_ray_stats
- jwql.database.database_interface.FGSDarkDarkCurrent¶
alias of
fgs_dark_dark_current
- jwql.database.database_interface.FGSDarkPixelStats¶
alias of
fgs_dark_pixel_stats
- jwql.database.database_interface.FGSDarkQueryHistory¶
alias of
fgs_dark_query_history
- jwql.database.database_interface.FGSEDBBlockStats¶
alias of
fgs_edb_blocks_stats
- jwql.database.database_interface.FGSEDBDailyStats¶
alias of
fgs_edb_daily_stats
- jwql.database.database_interface.FGSEDBEveryChangeStats¶
alias of
fgs_edb_every_change_stats
- jwql.database.database_interface.FGSEDBTimeIntervalStats¶
alias of
fgs_edb_time_interval_stats
- jwql.database.database_interface.FGSReadnoiseQueryHistory¶
alias of
fgs_readnoise_query_history
- jwql.database.database_interface.FGSReadnoiseStats¶
alias of
fgs_readnoise_stats
- class jwql.database.database_interface.FilesystemCharacteristics(**kwargs)[source]¶
ORM for table containing instrument-specific lists of the number of obervations corresponding to various instrument characteristics (e.g. filters)
- date¶
- filter_pupil¶
- id¶
- instrument¶
- obs_per_filter_pupil¶
- class jwql.database.database_interface.FilesystemGeneral(**kwargs)[source]¶
ORM for the general (non instrument specific) filesystem monitor table
- available¶
- date¶
- fits_file_count¶
- fits_file_size¶
- id¶
- total_file_count¶
- total_file_size¶
- used¶
- class jwql.database.database_interface.FilesystemInstrument(**kwargs)[source]¶
ORM for the instrument specific filesystem monitor table
- property colnames¶
A list of all column names in this table EXCEPT the date column
- count¶
- date¶
- filetype¶
- id¶
- instrument¶
- size¶
- jwql.database.database_interface.MIRIAnomaly¶
alias of
miri_anomaly
- jwql.database.database_interface.MIRIBadPixelQueryHistory¶
alias of
miri_bad_pixel_query_history
- jwql.database.database_interface.MIRIBadPixelStats¶
alias of
miri_bad_pixel_stats
- jwql.database.database_interface.MIRICosmicRayQueryHistory¶
alias of
miri_cosmic_ray_query_history
- jwql.database.database_interface.MIRICosmicRayStats¶
alias of
miri_cosmic_ray_stats
- jwql.database.database_interface.MIRIDarkDarkCurrent¶
alias of
miri_dark_dark_current
- jwql.database.database_interface.MIRIDarkPixelStats¶
alias of
miri_dark_pixel_stats
- jwql.database.database_interface.MIRIDarkQueryHistory¶
alias of
miri_dark_query_history
- jwql.database.database_interface.MIRIEDBBlockStats¶
alias of
miri_edb_blocks_stats
- jwql.database.database_interface.MIRIEDBDailyStats¶
alias of
miri_edb_daily_stats
- jwql.database.database_interface.MIRIEDBEveryChangeStats¶
alias of
miri_edb_every_change_stats
- jwql.database.database_interface.MIRIEDBTimeIntervalStats¶
alias of
miri_edb_time_interval_stats
- jwql.database.database_interface.MIRIReadnoiseQueryHistory¶
alias of
miri_readnoise_query_history
- jwql.database.database_interface.MIRIReadnoiseStats¶
alias of
miri_readnoise_stats
- class jwql.database.database_interface.Monitor(**kwargs)[source]¶
ORM for the
monitor
table- end_time¶
- id¶
- log_file¶
- monitor_name¶
- start_time¶
- status¶
- jwql.database.database_interface.NIRCamAnomaly¶
alias of
nircam_anomaly
- jwql.database.database_interface.NIRCamBadPixelQueryHistory¶
alias of
nircam_bad_pixel_query_history
- jwql.database.database_interface.NIRCamBadPixelStats¶
alias of
nircam_bad_pixel_stats
- jwql.database.database_interface.NIRCamBiasQueryHistory¶
alias of
nircam_bias_query_history
- jwql.database.database_interface.NIRCamBiasStats¶
alias of
nircam_bias_stats
- jwql.database.database_interface.NIRCamClawQueryHistory¶
alias of
nircam_claw_query_history
- jwql.database.database_interface.NIRCamClawStats¶
alias of
nircam_claw_stats
- jwql.database.database_interface.NIRCamCosmicRayQueryHistory¶
alias of
nircam_cosmic_ray_query_history
- jwql.database.database_interface.NIRCamCosmicRayStats¶
alias of
nircam_cosmic_ray_stats
- jwql.database.database_interface.NIRCamDarkDarkCurrent¶
alias of
nircam_dark_dark_current
- jwql.database.database_interface.NIRCamDarkPixelStats¶
alias of
nircam_dark_pixel_stats
- jwql.database.database_interface.NIRCamDarkQueryHistory¶
alias of
nircam_dark_query_history
- jwql.database.database_interface.NIRCamEDBBlockStats¶
alias of
nircam_edb_blocks_stats
- jwql.database.database_interface.NIRCamEDBDailyStats¶
alias of
nircam_edb_daily_stats
- jwql.database.database_interface.NIRCamEDBEveryChangeStats¶
alias of
nircam_edb_every_change_stats
- jwql.database.database_interface.NIRCamEDBTimeIntervalStats¶
alias of
nircam_edb_time_interval_stats
- jwql.database.database_interface.NIRCamReadnoiseQueryHistory¶
alias of
nircam_readnoise_query_history
- jwql.database.database_interface.NIRCamReadnoiseStats¶
alias of
nircam_readnoise_stats
- jwql.database.database_interface.NIRISSAnomaly¶
alias of
niriss_anomaly
- jwql.database.database_interface.NIRISSBadPixelQueryHistory¶
alias of
niriss_bad_pixel_query_history
- jwql.database.database_interface.NIRISSBadPixelStats¶
alias of
niriss_bad_pixel_stats
- jwql.database.database_interface.NIRISSBiasQueryHistory¶
alias of
niriss_bias_query_history
- jwql.database.database_interface.NIRISSBiasStats¶
alias of
niriss_bias_stats
- jwql.database.database_interface.NIRISSCosmicRayQueryHistory¶
alias of
niriss_cosmic_ray_query_history
- jwql.database.database_interface.NIRISSCosmicRayStats¶
alias of
niriss_cosmic_ray_stats
- jwql.database.database_interface.NIRISSDarkDarkCurrent¶
alias of
niriss_dark_dark_current
- jwql.database.database_interface.NIRISSDarkPixelStats¶
alias of
niriss_dark_pixel_stats
- jwql.database.database_interface.NIRISSDarkQueryHistory¶
alias of
niriss_dark_query_history
- jwql.database.database_interface.NIRISSEDBBlockStats¶
alias of
niriss_edb_blocks_stats
- jwql.database.database_interface.NIRISSEDBDailyStats¶
alias of
niriss_edb_daily_stats
- jwql.database.database_interface.NIRISSEDBEveryChangeStats¶
alias of
niriss_edb_every_change_stats
- jwql.database.database_interface.NIRISSEDBTimeIntervalStats¶
alias of
niriss_edb_time_interval_stats
- jwql.database.database_interface.NIRISSReadnoiseQueryHistory¶
alias of
niriss_readnoise_query_history
- jwql.database.database_interface.NIRISSReadnoiseStats¶
alias of
niriss_readnoise_stats
- jwql.database.database_interface.NIRSpecAnomaly¶
alias of
nirspec_anomaly
- jwql.database.database_interface.NIRSpecBadPixelQueryHistory¶
alias of
nirspec_bad_pixel_query_history
- jwql.database.database_interface.NIRSpecBadPixelStats¶
alias of
nirspec_bad_pixel_stats
- jwql.database.database_interface.NIRSpecBiasQueryHistory¶
alias of
nirspec_bias_query_history
- jwql.database.database_interface.NIRSpecBiasStats¶
alias of
nirspec_bias_stats
- jwql.database.database_interface.NIRSpecCosmicRayQueryHistory¶
alias of
nirspec_cosmic_ray_query_history
- jwql.database.database_interface.NIRSpecCosmicRayStats¶
alias of
nirspec_cosmic_ray_stats
- jwql.database.database_interface.NIRSpecDarkDarkCurrent¶
alias of
nirspec_dark_dark_current
- jwql.database.database_interface.NIRSpecDarkPixelStats¶
alias of
nirspec_dark_pixel_stats
- jwql.database.database_interface.NIRSpecDarkQueryHistory¶
alias of
nirspec_dark_query_history
- jwql.database.database_interface.NIRSpecEDBBlockStats¶
alias of
nirspec_edb_blocks_stats
- jwql.database.database_interface.NIRSpecEDBDailyStats¶
alias of
nirspec_edb_daily_stats
- jwql.database.database_interface.NIRSpecEDBEveryChangeStats¶
alias of
nirspec_edb_every_change_stats
- jwql.database.database_interface.NIRSpecEDBTimeIntervalStats¶
alias of
nirspec_edb_time_interval_stats
- jwql.database.database_interface.NIRSpecReadnoiseQueryHistory¶
alias of
nirspec_readnoise_query_history
- jwql.database.database_interface.NIRSpecReadnoiseStats¶
alias of
nirspec_readnoise_stats
- jwql.database.database_interface.NIRSpecTAQueryHistory¶
alias of
nirspec_ta_query_history
- jwql.database.database_interface.NIRSpecTAStats¶
alias of
nirspec_ta_stats
- jwql.database.database_interface.anomaly_orm_factory(class_name)[source]¶
Create a
SQLAlchemy
ORM Class for an anomaly table.- Parameters:
- class_namestr
The name of the class to be created
- Returns:
- classobj
The
SQLAlchemy
ORM
- jwql.database.database_interface.get_monitor_columns(data_dict, table_name)[source]¶
Read in the corresponding table definition text file to generate
SQLAlchemy
columns for the table.- Parameters:
- data_dictdict
A dictionary whose keys are column names and whose values are column definitions.
- table_namestr
The name of the database table
- Returns:
- data_dictdict
An updated
data_dict
with the approriate columns for the monitor added.
- jwql.database.database_interface.get_monitor_table_constraints(data_dict, table_name)[source]¶
Add any necessary table constrains to the given table via the
data_dict
.- Parameters:
- data_dictdict
A dictionary whose keys are column names and whose values are column definitions.
- table_namestr
The name of the database table
- Returns:
- data_dictdict
An updated
data_dict
with the approriate table constraints for the monitor added.
- jwql.database.database_interface.get_unique_values_per_column(table, column_name)[source]¶
Return a list of the unique values from a particular column in the given table.
- Parameters:
- tablesqlalchemy.orm.decl_api.DeclarativeMeta
SQL table to be searched. (e.g. table = eval(‘NIRCamDarkPixelStats’))
- column_namestr
Column name within the table to query
- Returns:
- distinct_colvalslist
List of unique values in the given column
- jwql.database.database_interface.load_connection(connection_string)[source]¶
Return
session
,base
,engine
, andmetadata
objects for connecting to thejwqldb
database.Create an
engine
using an givenconnection_string
. Create abase
class andsession
class from theengine
. Create an instance of thesession
class. Return thesession
,base
, andengine
instances. This was stolen from the ascql repository.- Parameters:
- connection_stringstr
A postgresql database connection string. The connection string should take the form:
dialect+driver://username:password@host:port/database
- Returns:
- sessionsesson object
Provides a holding zone for all objects loaded or associated with the database.
- basebase object
Provides a base class for declarative class definitions.
- engineengine object
Provides a source of database connectivity and behavior.
- meta: metadata object
The connection metadata
References
reset_database.py¶
Reset all tables in the jwqldb
database.
Authors¶
Matthew Bourque
Use¶
This script is intended to be used in the command line:
python reset_database.py
Dependencies¶
Users must have a
config.json
configuration file with a properconnection_string
key that points to thejwqldb
database. Theconnection_string
format ispostgresql+psycopg2://user:password@host:port/database
.