Scope 25: Site Survey Utility

Scope 25: Site Survey Utility#

  • Scope ID: 25

  • Owner: Charles Winans

  • Duration: 40 Hours

  • Date: 10/10/2024

  • Due: 10/12/2024

  • Actual Completion Date: 10/27/2024

Overview#

Create modular utility to run a minimal workign version of a site survey while remaining consistent with existing metacrud standards and adapting standards as neccesary.

Justification#

Site survey has been cumbersome and far from self-service requiring developer assistance through the lack of design and implementation completion. This is the most expedient path to this goal of a continuously operable service.

Deliverables#

Complete the following steps: 1. Database docker container w/ running PostGreSQL database 2. Server docker container w/ running Python utility on a loop that restarts every 5 seconds 3. PostGreSQL connection details in /etc directory 4. R2 connection details in /etc directory 5. Find site survey manifests within bucket and add to database 6. Find directories within bucket 7. Find subdirectories/PDF sets within site survey

  1. Conenct the python application to the postgresql database

  1. Use credentials to look for new manifest files

  2. Index manifests

    1. Organizations

    2. Directories(Make if missing)

    3. Subdirectories(Make if missing)

  1. Migration of existing schemas to PostGreSQL

  2. Migration of existing data to PostGreSQL

  3. Function: Read manifest file from directory

  4. Function: Locate and index all directories and subdirectories within site survey

  5. Function: Read all individual pages

  6. Function: Read all sets and break into pages

  7. Function: Add organization to database

  8. Function: Add sets to database

  9. Function: Add pages to database

  10. Query: Produce metadata for set

  11. Query: Produce metadata for page

  12. Function: Create set cover

  13. Function: Create all set covers

  14. Function: Create page cover

  15. Function: Assemble sets with pages and page labels

  16. Function: Output all

  1. Python function to read manifest file as currently organized and populate shcemas with information and relations between objects within schemas for organization, set, page, version, etc.

  2. Library of set and page label sheets

  3. Query to produce JSON metadata for set and page labels

  4. Generation of set label

  5. Generation of page pabel

  6. Separation of PDf sets into pages

  7. Insertion of covers and pages into sets

  8. Assembly of sets into complete set

Potential Future Scopes of Work#

1. Python application cleanup, standardization, and documentation 1. API specification site 2. OpenAPI specification based on src/metacrud2/openapi.json 3. OpenAPI compliance server 4. Proxy OpenAPI compliant server via nginx and via cloudflare w/ service acct auth 4. Proxy OpenAPI compliant server via nginx and via cloudflare w/ auth0 auth