database

database_interface.py

A module to interact with the JWQL postgresql database jwqldb

The load_connection() function within this module allows the user to connect to the jwqldb database via the session, base, and engine objects (described below). The classes within serve as ORMs (Object-relational mappings) that define the individual tables of the relational database.

The engine object serves as the low-level database API and perhaps most importantly contains dialects which allows the sqlalchemy module to communicate with the database.

The base object serves as a base class for class definitions. It produces Table objects and constructs ORMs.

The session object manages operations on ORM-mapped objects, as construced by the base. These operations include querying, for example.

Authors

  • Joe Filippazzo

  • Johannes Sahlmann

  • Matthew Bourque

  • Lauren Chambers

  • Bryan Hilbert

  • Misty Cracraft

  • Sara Ogaz

Use

Executing the module on the command line will build the database tables defined within:

python database_interface.py

Users wishing to interact with the existing database may do so by importing various connection objects and database tables, for example:

from jwql.database.database_interface import Anomaly
from jwql.database.database_interface import session

results = session.query(Anomaly).all()

Dependencies

The user must have a configuration file named config.json placed in the jwql directory.

class jwql.database.database_interface.CentralStore(**kwargs)[source]

ORM for the central storage area filesystem monitor table

area
available
date
id
size
used
jwql.database.database_interface.FGSAnomaly

alias of jwql.database.database_interface.fgs_anomaly

jwql.database.database_interface.FGSBadPixelQueryHistory

alias of jwql.database.database_interface.fgs_bad_pixel_query_history

jwql.database.database_interface.FGSBadPixelStats

alias of jwql.database.database_interface.fgs_bad_pixel_stats

jwql.database.database_interface.FGSDarkDarkCurrent

alias of jwql.database.database_interface.fgs_dark_dark_current

jwql.database.database_interface.FGSDarkPixelStats

alias of jwql.database.database_interface.fgs_dark_pixel_stats

jwql.database.database_interface.FGSDarkQueryHistory

alias of jwql.database.database_interface.fgs_dark_query_history

jwql.database.database_interface.FGSReadnoiseQueryHistory

alias of jwql.database.database_interface.fgs_readnoise_query_history

jwql.database.database_interface.FGSReadnoiseStats

alias of jwql.database.database_interface.fgs_readnoise_stats

class jwql.database.database_interface.FilesystemGeneral(**kwargs)[source]

ORM for the general (non instrument specific) filesystem monitor table

available
date
fits_file_count
fits_file_size
id
total_file_count
total_file_size
used
class jwql.database.database_interface.FilesystemInstrument(**kwargs)[source]

ORM for the instrument specific filesystem monitor table

property colnames

A list of all column names in this table EXCEPT the date column

count
date
filetype
id
instrument
size
jwql.database.database_interface.MIRIAnomaly

alias of jwql.database.database_interface.miri_anomaly

jwql.database.database_interface.MIRIBadPixelQueryHistory

alias of jwql.database.database_interface.miri_bad_pixel_query_history

jwql.database.database_interface.MIRIBadPixelStats

alias of jwql.database.database_interface.miri_bad_pixel_stats

jwql.database.database_interface.MIRIDarkDarkCurrent

alias of jwql.database.database_interface.miri_dark_dark_current

jwql.database.database_interface.MIRIDarkPixelStats

alias of jwql.database.database_interface.miri_dark_pixel_stats

jwql.database.database_interface.MIRIDarkQueryHistory

alias of jwql.database.database_interface.miri_dark_query_history

jwql.database.database_interface.MIRIReadnoiseQueryHistory

alias of jwql.database.database_interface.miri_readnoise_query_history

jwql.database.database_interface.MIRIReadnoiseStats

alias of jwql.database.database_interface.miri_readnoise_stats

class jwql.database.database_interface.Monitor(**kwargs)[source]

ORM for the monitor table

end_time
id
log_file
monitor_name
start_time
status
jwql.database.database_interface.NIRCamAnomaly

alias of jwql.database.database_interface.nircam_anomaly

jwql.database.database_interface.NIRCamBadPixelQueryHistory

alias of jwql.database.database_interface.nircam_bad_pixel_query_history

jwql.database.database_interface.NIRCamBadPixelStats

alias of jwql.database.database_interface.nircam_bad_pixel_stats

jwql.database.database_interface.NIRCamBiasQueryHistory

alias of jwql.database.database_interface.nircam_bias_query_history

jwql.database.database_interface.NIRCamBiasStats

alias of jwql.database.database_interface.nircam_bias_stats

jwql.database.database_interface.NIRCamDarkDarkCurrent

alias of jwql.database.database_interface.nircam_dark_dark_current

jwql.database.database_interface.NIRCamDarkPixelStats

alias of jwql.database.database_interface.nircam_dark_pixel_stats

jwql.database.database_interface.NIRCamDarkQueryHistory

alias of jwql.database.database_interface.nircam_dark_query_history

jwql.database.database_interface.NIRCamReadnoiseQueryHistory

alias of jwql.database.database_interface.nircam_readnoise_query_history

jwql.database.database_interface.NIRCamReadnoiseStats

alias of jwql.database.database_interface.nircam_readnoise_stats

jwql.database.database_interface.NIRISSAnomaly

alias of jwql.database.database_interface.niriss_anomaly

jwql.database.database_interface.NIRISSBadPixelQueryHistory

alias of jwql.database.database_interface.niriss_bad_pixel_query_history

jwql.database.database_interface.NIRISSBadPixelStats

alias of jwql.database.database_interface.niriss_bad_pixel_stats

jwql.database.database_interface.NIRISSBiasQueryHistory

alias of jwql.database.database_interface.niriss_bias_query_history

jwql.database.database_interface.NIRISSBiasStats

alias of jwql.database.database_interface.niriss_bias_stats

jwql.database.database_interface.NIRISSDarkDarkCurrent

alias of jwql.database.database_interface.niriss_dark_dark_current

jwql.database.database_interface.NIRISSDarkPixelStats

alias of jwql.database.database_interface.niriss_dark_pixel_stats

jwql.database.database_interface.NIRISSDarkQueryHistory

alias of jwql.database.database_interface.niriss_dark_query_history

jwql.database.database_interface.NIRISSReadnoiseQueryHistory

alias of jwql.database.database_interface.niriss_readnoise_query_history

jwql.database.database_interface.NIRISSReadnoiseStats

alias of jwql.database.database_interface.niriss_readnoise_stats

jwql.database.database_interface.NIRSpecAnomaly

alias of jwql.database.database_interface.nirspec_anomaly

jwql.database.database_interface.NIRSpecBadPixelQueryHistory

alias of jwql.database.database_interface.nirspec_bad_pixel_query_history

jwql.database.database_interface.NIRSpecBadPixelStats

alias of jwql.database.database_interface.nirspec_bad_pixel_stats

jwql.database.database_interface.NIRSpecBiasQueryHistory

alias of jwql.database.database_interface.nirspec_bias_query_history

jwql.database.database_interface.NIRSpecBiasStats

alias of jwql.database.database_interface.nirspec_bias_stats

jwql.database.database_interface.NIRSpecDarkDarkCurrent

alias of jwql.database.database_interface.nirspec_dark_dark_current

jwql.database.database_interface.NIRSpecDarkPixelStats

alias of jwql.database.database_interface.nirspec_dark_pixel_stats

jwql.database.database_interface.NIRSpecDarkQueryHistory

alias of jwql.database.database_interface.nirspec_dark_query_history

jwql.database.database_interface.NIRSpecReadnoiseQueryHistory

alias of jwql.database.database_interface.nirspec_readnoise_query_history

jwql.database.database_interface.NIRSpecReadnoiseStats

alias of jwql.database.database_interface.nirspec_readnoise_stats

jwql.database.database_interface.anomaly_orm_factory(class_name)[source]

Create a SQLAlchemy ORM Class for an anomaly table.

Parameters:
class_namestr

The name of the class to be created

Returns:
classobj

The SQLAlchemy ORM

jwql.database.database_interface.data_frame

Method to return a pandas.DataFrame of the results

jwql.database.database_interface.get_monitor_columns(data_dict, table_name)[source]

Read in the corresponding table definition text file to generate SQLAlchemy columns for the table.

Parameters:
data_dictdict

A dictionary whose keys are column names and whose values are column definitions.

table_namestr

The name of the database table

Returns:
data_dictdict

An updated data_dict with the approriate columns for the monitor added.

jwql.database.database_interface.get_monitor_table_constraints(data_dict, table_name)[source]

Add any necessary table constrains to the given table via the data_dict.

Parameters:
data_dictdict

A dictionary whose keys are column names and whose values are column definitions.

table_namestr

The name of the database table

Returns:
data_dictdict

An updated data_dict with the approriate table constraints for the monitor added.

jwql.database.database_interface.load_connection(connection_string)[source]

Return session, base, engine, and metadata objects for connecting to the jwqldb database.

Create an engine using an given connection_string. Create a base class and session class from the engine. Create an instance of the session class. Return the session, base, and engine instances. This was stolen from the ascql repository.

Parameters:
connection_stringstr

A postgresql database connection string. The connection string should take the form: dialect+driver://username:password@host:port/database

Returns:
sessionsesson object

Provides a holding zone for all objects loaded or associated with the database.

basebase object

Provides a base class for declarative class definitions.

engineengine object

Provides a source of database connectivity and behavior.

meta: metadata object

The connection metadata

References

ascql:

https://github.com/spacetelescope/acsql/blob/master/acsql/database/database_interface.py

jwql.database.database_interface.monitor_orm_factory(class_name)[source]

Create a SQLAlchemy ORM Class for a jwql instrument monitor.

Parameters:
class_namestr

The name of the class to be created

Returns:
classobj

The SQLAlchemy ORM

jwql.database.database_interface.set_read_permissions()[source]

Set read permissions for db tables

reset_database.py

Reset all tables in the jwqldb database.

Authors

  • Matthew Bourque

Use

This script is intended to be used in the command line:

python reset_database.py

Dependencies

Users must have a config.json configuration file with a proper connection_string key that points to the jwqldb database. The connection_string format is postgresql+psycopg2://user:password@host:port/database.