V1.1-Preview

added inbound and outbound API, and validation, console, and upload UI
This commit is contained in:
Joshua Hare 2025-04-11 00:12:23 +10:00
parent 1169e3b16d
commit 097fefdfe7
19 changed files with 1304 additions and 211 deletions

View File

@ -12,6 +12,7 @@ COPY app.py .
COPY services.py . COPY services.py .
COPY templates/ templates/ COPY templates/ templates/
COPY static/ static/ COPY static/ static/
COPY tests/ tests/
# Ensure /tmp is writable as a fallback # Ensure /tmp is writable as a fallback
RUN mkdir -p /tmp && chmod 777 /tmp RUN mkdir -p /tmp && chmod 777 /tmp

377
README.md
View File

@ -2,17 +2,33 @@
## Overview ## Overview
FHIRFLARE IG Toolkit is a Flask-based web application designed to simplify the management of FHIR Implementation Guides (IGs). It allows users to import, process, view, and manage FHIR packages, with features to handle duplicate dependencies and visualize processed IGs. The toolkit is built to assist developers, researchers, and healthcare professionals working with FHIR standards. The FHIRFLARE IG Toolkit is a Flask-based web application designed to simplify the management, processing, and deployment of FHIR Implementation Guides (IGs). It allows users to import IG packages, process them to extract resource types and profiles, and push them to a FHIR server. The application features a user-friendly interface with a live console for real-time feedback during operations like pushing IGs to a FHIR server.
This tool was initially developed as an IG package viewer within a larger project I was building. As the requirements expanded and it became clear there was a strong need for a lightweight, purpose-built solution to interact with IG packages, I decided to decouple it from the original codebase and release it as a standalone open-source utility for broader community use. ## Features
### Key Features - **Import IGs**: Download FHIR IG packages and their dependencies from a package registry.
- **Import FHIR Packages**: Download FHIR IGs and their dependencies by specifying package names and versions. - **Manage IGs**: View, process, and delete downloaded IGs, with duplicate detection.
- **Manage Duplicates**: Detect and highlight duplicate packages with different versions, using color-coded indicators. - **Process IGs**: Extract resource types, profiles, must-support elements, and examples from IGs.
- **Process IGs**: Extract and process FHIR resources, including structure definitions, must-support elements, and examples. - **Push IGs**: Upload IG resources to a FHIR server with real-time console output.
- **View Details**: Explore processed IGs with detailed views of resource types and examples. - **API Support**: Provides RESTful API endpoints for importing and pushing IGs.
- **Database Integration**: Store processed IGs in a SQLite database for persistence. - **Live Console**: Displays real-time logs during push operations.
- **User-Friendly Interface**: Built with Bootstrap for a responsive and intuitive UI.
## Technology Stack
The FHIRFLARE IG Toolkit is built using the following technologies:
- **Python 3.9+**: Core programming language for the backend.
- **Flask 2.0+**: Lightweight web framework for building the application.
- **Flask-SQLAlchemy**: ORM for managing the SQLite database.
- **Flask-WTF**: Handles form creation and CSRF protection.
- **Jinja2**: Templating engine for rendering HTML pages.
- **Bootstrap 5**: Frontend framework for responsive UI design.
- **JavaScript (ES6)**: Client-side scripting for interactive features like the live console.
- **SQLite**: Lightweight database for storing processed IG metadata.
- **Docker**: Containerization for consistent deployment.
- **Requests**: Python library for making HTTP requests to FHIR servers.
- **Tarfile**: Python library for handling `.tgz` package files.
- **Logging**: Python's built-in logging for debugging and monitoring.
## Technology Stack ## Technology Stack
@ -37,203 +53,212 @@ This application is built using the following technologies:
## Prerequisites ## Prerequisites
Before setting up the project, ensure you have the following installed: - **Python 3.9+**: Ensure Python is installed on your system.
- **Python 3.8+** - **Docker**: Required for containerized deployment.
- **Docker** (optional, for containerized deployment) - **pip**: Python package manager for installing dependencies.
- **pip** (Python package manager)
## Setup Instructions ## Setup Instructions
### 1. Clone the Repository 1. **Clone the Repository**:
```bash
git clone https://github.com/your-username/FLARE-FHIR-IG-Toolkit.git
cd FLARE-FHIR-IG-Toolkit
```
### 2. Create a Virtual Environment
```bash
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
```
### 3. Install Dependencies
Install the required Python packages using `pip`:
```bash
pip install -r requirements.txt
```
If you dont have a `requirements.txt` file, you can install the core dependencies manually:
```bash
pip install flask flask-sqlalchemy flask-wtf
```
### 4. Set Up the Instance Directory
The application uses an `instance` directory to store the SQLite database and FHIR packages. Create this directory and set appropriate permissions:
```bash
mkdir instance
mkdir instance/fhir_packages
chmod -R 777 instance # Ensure the directory is writable
```
### 5. Initialize the Database
The application uses a SQLite database (`instance/fhir_ig.db`) to store processed IGs. The database is automatically created when you first run the application, but you can also initialize it manually:
```bash
python -c "from app import db; db.create_all()"
```
### 6. Run the Application Locally
Start the Flask development server:
```bash
python app.py
```
The application will be available at `http://localhost:5000`.
### 7. (Optional) Run with Docker
If you prefer to run the application in a Docker container:
```bash
docker build -t flare-fhir-ig-toolkit .
docker run -p 5000:5000 -v $(pwd)/instance:/app/instance flare-fhir-ig-toolkit
```
The application will be available at `http://localhost:5000`.
## Database Management
The application uses a SQLite database (`instance/fhir_ig.db`) to store processed IGs. Below are steps to create, purge, and recreate the database.
### Creating the Database
The database is automatically created when you first run the application. To create it manually:
```bash
python -c "from app import db; db.create_all()"
```
This will create `instance/fhir_ig.db` with the necessary tables.
### Purging the Database
To purge the database (delete all data while keeping the schema):
1. Stop the application if its running.
2. Run the following command to drop all tables and recreate them:
```bash ```bash
python -c "from app import db; db.drop_all(); db.create_all()" git clone <repository-url>
``` cd fhirflare-ig-toolkit
3. Alternatively, you can delete the database file and recreate it:
```bash
rm instance/fhir_ig.db
python -c "from app import db; db.create_all()"
``` ```
### Recreating the Database 2. **Install Dependencies**:
To completely recreate the database (e.g., after making schema changes): Create a virtual environment and install the required packages:
1. Stop the application if its running.
2. Delete the existing database file:
```bash ```bash
rm instance/fhir_ig.db python -m venv venv
``` source venv/bin/activate # On Windows: venv\Scripts\activate
3. Recreate the database: pip install -r requirements.txt
```bash
python -c "from app import db; db.create_all()"
```
4. Restart the application:
```bash
python app.py
``` ```
**Note**: Ensure the `instance` directory has write permissions (`chmod -R 777 instance`) to avoid permission errors when creating or modifying the database. 3. **Set Environment Variables**:
Set the `FLASK_SECRET_KEY` and `API_KEY` environment variables for security:
```bash
export FLASK_SECRET_KEY='your-secure-secret-key'
export API_KEY='your-api-key'
```
4. **Run the Application Locally**:
Start the Flask development server:
```bash
flask run
```
The application will be available at `http://localhost:5000`.
5. **Run with Docker**:
Build and run the application using Docker:
```bash
docker build -t flare-fhir-ig-toolkit .
docker run -p 5000:5000 -e FLASK_SECRET_KEY='your-secure-secret-key' -e API_KEY='your-api-key' -v $(pwd)/instance:/app/instance flare-fhir-ig-toolkit
```
Access the application at `http://localhost:5000`.
## Usage ## Usage
### Importing FHIR Packages 1. **Import an IG**:
1. Navigate to the "Import IGs" page (`/import-ig`). - Navigate to the "Import IG" tab.
2. Enter the package name (e.g., `hl7.fhir.us.core`) and version (e.g., `1.0.0` or `current`). - Enter a package name (e.g., `hl7.fhir.us.core`) and version (e.g., `3.1.1`).
3. Click "Fetch & Download IG" to download the package and its dependencies. - Click "Fetch & Download IG" to download the package and its dependencies.
4. Youll be redirected to the "Manage FHIR Packages" page to view the downloaded packages.
### Managing FHIR Packages 2. **Manage IGs**:
- **View Downloaded Packages**: The "Manage FHIR Packages" page (`/view-igs`) lists all downloaded packages. - Go to the "Manage FHIR Packages" tab to view downloaded IGs.
- **Handle Duplicates**: Duplicate packages with different versions are highlighted with color-coded rows (e.g., yellow for one group, light blue for another). - Process, delete, or view details of IGs. Duplicates are highlighted for resolution.
- **Process Packages**: Click "Process" to extract and store package details in the database.
- **Delete Packages**: Click "Delete" to remove a package from the filesystem.
### Viewing Processed IGs 3. **Push IGs to a FHIR Server**:
- Processed packages are listed in the "Processed Packages" section. - Navigate to the "Push IGs" tab.
- Click "View" to see detailed information about a processed IG, including resource types and examples. - Select a package, enter a FHIR server URL (e.g., `http://hapi.fhir.org/baseR4`), and choose whether to include dependencies.
- Click "Unload" to remove a processed IG from the database. - Click "Push to FHIR Server" to upload resources, with progress shown in the live console.
## Project Structure 4. **API Usage**:
- **Import IG**: `POST /api/import-ig`
```bash
curl -X POST http://localhost:5000/api/import-ig \
-H "Content-Type: application/json" \
-d '{"package_name": "hl7.fhir.us.core", "version": "3.1.1", "api_key": "your-api-key"}'
```
- **Push IG**: `POST /api/push-ig`
```bash
curl -X POST http://localhost:5000/api/push-ig \
-H "Content-Type: application/json" \
-H "Accept: application/x-ndjson" \
-d '{"package_name": "hl7.fhir.us.core", "version": "3.1.1", "fhir_server_url": "http://hapi.fhir.org/baseR4", "include_dependencies": true, "api_key": "your-api-key"}'
```
``` ## Testing
FLARE-FHIR-IG-Toolkit/
├── app.py # Main Flask application The project includes a comprehensive test suite to ensure the reliability of the application. Tests cover the UI, API endpoints, database operations, file handling, and security features like CSRF protection.
├── instance/ # Directory for SQLite database and FHIR packages
│ ├── fhir_ig.db # SQLite database ### Test Prerequisites
│ └── fhir_packages/ # Directory for downloaded FHIR packages
├── static/ # Static files (e.g., favicon.ico, FHIRFLARE.png) - **pytest**: For running the tests.
│ ├── FHIRFLARE.png - **unittest-mock**: For mocking dependencies in tests.
│ └── favicon.ico
├── templates/ # HTML templates Install the test dependencies:
│ ├── base.html ```bash
│ ├── cp_downloaded_igs.html pip install pytest unittest-mock
│ ├── cp_view_processed_ig.html
│ ├── import_ig.html
│ └── index.html
├── services.py # Helper functions for processing FHIR packages
└── README.md # Project documentation
``` ```
## Development Notes ### Running Tests
### Background 1. **Navigate to the Project Root**:
The FHIRFLARE IG Toolkit was developed to address the need for a user-friendly tool to manage FHIR Implementation Guides. The project focuses on providing a seamless experience for importing, processing, and analyzing FHIR packages, with a particular emphasis on handling duplicate dependencies—a common challenge in FHIR development. ```bash
cd /path/to/fhirflare-ig-toolkit
```
### Technical Decisions 2. **Run the Tests**:
- **Flask**: Chosen for its lightweight and flexible nature, making it ideal for a small to medium-sized web application. Run the test suite using `pytest`:
- **SQLite**: Used as the database for simplicity and ease of setup. For production use, consider switching to a more robust database like PostgreSQL. ```bash
- **Bootstrap**: Integrated for a responsive and professional UI, with custom CSS to handle duplicate package highlighting. pytest tests/test_app.py -v
- **Docker Support**: Added to simplify deployment and ensure consistency across development and production environments. ```
- The `-v` flag provides verbose output, showing the status of each test.
- Alternatively, run from the `tests/` directory with:
```bash
cd tests/
pytest test_app.py -v
```
### Known Issues and Workarounds ### Test Coverage
- **Bootstrap CSS Conflicts**: Early versions of the application had issues with Bootstraps table background styles (`--bs-table-bg`) overriding custom row colors for duplicate packages. This was resolved by setting `--bs-table-bg` to `transparent` for the affected table (see `templates/cp_downloaded_igs.html`).
- **Database Permissions**: The `instance` directory must be writable by the application. If you encounter permission errors, ensure the directory has the correct permissions (`chmod -R 777 instance`).
- **Package Parsing**: Some FHIR package filenames may not follow the expected `name-version.tgz` format, leading to parsing issues. The application includes a fallback to treat such files as name-only packages, but this may need further refinement.
### Future Improvements The test suite includes 27 test cases covering the following areas:
- **Sorting Versions**: Add sorting for package versions in the "Manage FHIR Packages" view to display them in a consistent order (e.g., ascending or descending).
- **Advanced Duplicate Handling**: Implement options to resolve duplicates (e.g., keep the latest version, merge resources). - **UI Pages**:
- **Production Database**: Support for PostgreSQL or MySQL for better scalability in production environments. - Homepage (`/`): Rendering and content.
- **Testing**: Add unit tests using `pytest` to cover core functionality, especially package processing and database operations. - Import IG page (`/import-ig`): Form rendering and submission (success, failure, invalid input).
- **Inbound API for IG Packages**: Develop API endpoints to allow external tools to push IG packages to FHIRFLARE. The API should automatically resolve dependencies, return a list of dependencies, and identify any duplicate dependencies. For example: - Manage IGs page (`/view-igs`): Rendering with and without packages.
- Endpoint: `POST /api/import-ig` - Push IGs page (`/push-igs`): Rendering and live console.
- Request: `{ "package_name": "hl7.fhir.us.core", "version": "1.0.0" }` - View Processed IG page (`/view-ig/<id>`): Rendering processed IG details.
- Response: `{ "status": "success", "dependencies": ["hl7.fhir.r4.core#4.0.1"], "duplicates": ["hl7.fhir.r4.core#4.0.1 (already exists as 5.0.0)"] }`
- **Outbound API for Pushing IGs to FHIR Servers**: Create an outbound API to push a chosen IG (with its dependencies) to a FHIR server, or allow pushing a single IG without dependencies. The API should process the servers responses and provide feedback. For example: - **API Endpoints**:
- Endpoint: `POST /api/push-ig` - `POST /api/import-ig`: Success, invalid API key, missing parameters.
- Request: `{ "package_name": "hl7.fhir.us.core", "version": "1.0.0", "fhir_server_url": "https://fhir-server.example.com", "include_dependencies": true }` - `POST /api/push-ig`: Success, invalid API key, package not found.
- Response: `{ "status": "success", "pushed_packages": ["hl7.fhir.us.core#1.0.0", "hl7.fhir.r4.core#4.0.1"], "server_response": "Resources uploaded successfully" }` - `GET /get-structure`: Fetching structure definitions (success, not found).
- **Far-Distant Improvements**: - `GET /get-example`: Fetching example content (success, invalid path).
- **Cache Service for IGs**: Implement a cache service to store all IGs, allowing for quick querying of package metadata without reprocessing. This could use an in-memory store like Redis to improve performance.
- **Database Index Optimization**: Modify the database structure to use a composite index on `package_name` and `version` (e.g., `ProcessedIg.package_name + ProcessedIg.version` as a unique key). This would allow the `/view-igs` page and API endpoints to directly query specific packages (e.g., `/api/ig/hl7.fhir.us.core/1.0.0`) without scanning the entire table. - **Database Operations**:
- Processing IGs: Storing processed IG data in the database.
- Unloading IGs: Removing processed IG records.
- Viewing processed IGs: Retrieving and displaying processed IG data.
- **File Operations**:
- Processing IG packages: Extracting data from `.tgz` files.
- Deleting IG packages: Removing `.tgz` files from the filesystem.
- **Security**:
- Secret Key: CSRF protection for form submissions.
- Flash Messages: Session integrity for flash messages.
### Example Test Output
A successful test run will look like this:
```
================================================================ test session starts =================================================================
platform linux -- Python 3.9.22, pytest-8.3.5, pluggy-1.5.0 -- /usr/local/bin/python3.9
cachedir: .pytest_cache
rootdir: /app/tests
collected 27 items
test_app.py::TestFHIRFlareIGToolkit::test_homepage PASSED [ 3%]
test_app.py::TestFHIRFlareIGToolkit::test_import_ig_page PASSED [ 7%]
test_app.py::TestFHIRFlareIGToolkit::test_import_ig_success PASSED [ 11%]
test_app.py::TestFHIRFlareIGToolkit::test_import_ig_failure PASSED [ 14%]
test_app.py::TestFHIRFlareIGToolkit::test_import_ig_invalid_input PASSED [ 18%]
test_app.py::TestFHIRFlareIGToolkit::test_view_igs_no_packages PASSED [ 22%]
test_app.py::TestFHIRFlareIGToolkit::test_view_igs_with_packages PASSED [ 25%]
test_app.py::TestFHIRFlareIGToolkit::test_process_ig_success PASSED [ 29%]
test_app.py::TestFHIRFlareIGToolkit::test_process_ig_invalid_file PASSED [ 33%]
test_app.py::TestFHIRFlareIGToolkit::test_delete_ig_success PASSED [ 37%]
test_app.py::TestFHIRFlareIGToolkit::test_delete_ig_file_not_found PASSED [ 40%]
test_app.py::TestFHIRFlareIGToolkit::test_unload_ig_success PASSED [ 44%]
test_app.py::TestFHIRFlareIGToolkit::test_unload_ig_invalid_id PASSED [ 48%]
test_app.py::TestFHIRFlareIGToolkit::test_view_processed_ig PASSED [ 51%]
test_app.py::TestFHIRFlareIGToolkit::test_push_igs_page PASSED [ 55%]
test_app.py::TestFHIRFlareIGToolkit::test_api_import_ig_success PASSED [ 59%]
test_app.py::TestFHIRFlareIGToolkit::test_api_import_ig_invalid_api_key PASSED [ 62%]
test_app.py::TestFHIRFlareIGToolkit::test_api_import_ig_missing_params PASSED [ 66%]
test_app.py::TestFHIRFlareIGToolkit::test_api_push_ig_success PASSED [ 70%]
test_app.py::TestFHIRFlareIGToolkit::test_api_push_ig_invalid_api_key PASSED [ 74%]
test_app.py::TestFHIRFlareIGToolkit::test_api_push_ig_package_not_found PASSED [ 77%]
test_app.py::TestFHIRFlareIGToolkit::test_secret_key_csrf PASSED [ 81%]
test_app.py::TestFHIRFlareIGToolkit::test_secret_key_flash_messages PASSED [ 85%]
test_app.py::TestFHIRFlareIGToolkit::test_get_structure_definition_success PASSED [ 88%]
test_app.py::TestFHIRFlareIGToolkit::test_get_structure_definition_not_found PASSED [ 92%]
test_app.py::TestFHIRFlareIGToolkit::test_get_example_content_success PASSED [ 96%]
test_app.py::TestFHIRFlareIGToolkit::test_get_example_content_invalid_path PASSED [100%]
============================================================= 27 passed in 1.23s ==============================================================
```
### Troubleshooting Tests
- **ModuleNotFoundError**: If you encounter `ModuleNotFoundError: No module named 'app'`, ensure youre running the tests from the project root (`/app/`) or that the `sys.path` modification in `test_app.py` is correctly adding the parent directory.
- **Missing Templates**: If tests fail with `TemplateNotFound`, ensure all required templates (`index.html`, `import_ig.html`, etc.) are in the `/app/templates/` directory.
- **Missing Dependencies**: If tests fail due to missing `services.py` or its functions, ensure `services.py` is present in `/app/` and contains the required functions (`import_package_and_dependencies`, `process_package_file`, etc.).
## Directory Structure
- `app.py`: Main Flask application file.
- `services.py`: Business logic for importing, processing, and pushing IGs.
- `templates/`: HTML templates for the UI.
- `instance/`: Directory for SQLite database and downloaded packages.
- `tests/`: Directory for test files.
- `test_app.py`: Test suite for the application.
- `requirements.txt`: List of Python dependencies.
- `Dockerfile`: Docker configuration for containerized deployment.
## Contributing ## Contributing
Contributions are welcome! To contribute: Contributions are welcome! Please fork the repository, create a feature branch, and submit a pull request with your changes.
1. Fork the repository.
2. Create a new branch (`git checkout -b feature/your-feature`).
3. Make your changes and commit them (`git commit -m "Add your feature"`).
4. Push to your branch (`git push origin feature/your-feature`).
5. Open a Pull Request.
Please ensure your code follows the projects coding style and includes appropriate tests.
## Troubleshooting
- **Database Issues**: If the SQLite database (`instance/fhir_ig.db`) cannot be created, ensure the `instance` directory is writable. You may need to adjust permissions (`chmod -R 777 instance`).
- **Package Download Fails**: Verify your internet connection and ensure the package name and version are correct.
- **Colors Not Displaying**: If table row colors for duplicates are not showing, inspect the page with browser developer tools (F12) to check for CSS conflicts with Bootstrap.
## License ## License
<<<<<<< Updated upstream
This project is licensed under the Apache License, Version 2.0. See the [LICENSE.md](LICENSE.md) file for details. This project is licensed under the Apache License, Version 2.0. See the [LICENSE.md](LICENSE.md) file for details.
## Contact ## Contact
For questions or support, please open an issue on GitHub or contact the maintainers at [your-email@example.com](mailto:your-email@example.com). For questions or support, please open an issue on GitHub or contact the maintainers at [your-email@example.com](mailto:your-email@example.com).
=======
This project is licensed under the Apache 2.0 License. See the `LICENSE` file for details.
>>>>>>> Stashed changes

351
app.py
View File

@ -1,4 +1,4 @@
from flask import Flask, render_template, render_template_string, request, redirect, url_for, flash, jsonify from flask import Flask, render_template, render_template_string, request, redirect, url_for, flash, jsonify, Response
from flask_sqlalchemy import SQLAlchemy from flask_sqlalchemy import SQLAlchemy
from flask_wtf import FlaskForm from flask_wtf import FlaskForm
from wtforms import StringField, SubmitField from wtforms import StringField, SubmitField
@ -9,6 +9,8 @@ import json
from datetime import datetime from datetime import datetime
import services import services
import logging import logging
import requests
import re # Added for regex validation
# Set up logging # Set up logging
logging.basicConfig(level=logging.DEBUG) logging.basicConfig(level=logging.DEBUG)
@ -19,6 +21,7 @@ app.config['SECRET_KEY'] = 'your-secret-key-here'
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////app/instance/fhir_ig.db' app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////app/instance/fhir_ig.db'
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False
app.config['FHIR_PACKAGES_DIR'] = os.path.join(app.instance_path, 'fhir_packages') app.config['FHIR_PACKAGES_DIR'] = os.path.join(app.instance_path, 'fhir_packages')
app.config['API_KEY'] = 'your-api-key-here' # Hardcoded API key for now; replace with a secure solution
# Ensure directories exist and are writable # Ensure directories exist and are writable
instance_path = '/app/instance' instance_path = '/app/instance'
@ -46,7 +49,7 @@ db = SQLAlchemy(app)
class IgImportForm(FlaskForm): class IgImportForm(FlaskForm):
package_name = StringField('Package Name (e.g., hl7.fhir.us.core)', validators=[ package_name = StringField('Package Name (e.g., hl7.fhir.us.core)', validators=[
DataRequired(), DataRequired(),
Regexp(r'^[a-zA-Z0-9]+(\.[a-zA-Z0-9]+)+$', message='Invalid package name format.') Regexp(r'^[a-zAZ0-9]+(\.[a-zA-Z0-9]+)+$', message='Invalid package name format.')
]) ])
package_version = StringField('Package Version (e.g., 1.0.0 or current)', validators=[ package_version = StringField('Package Version (e.g., 1.0.0 or current)', validators=[
DataRequired(), DataRequired(),
@ -63,6 +66,18 @@ class ProcessedIg(db.Model):
must_support_elements = db.Column(db.JSON, nullable=True) must_support_elements = db.Column(db.JSON, nullable=True)
examples = db.Column(db.JSON, nullable=True) examples = db.Column(db.JSON, nullable=True)
# Middleware to check API key
def check_api_key():
api_key = request.json.get('api_key') if request.is_json else None
if not api_key:
logger.error("API key missing in request")
return jsonify({"status": "error", "message": "API key missing"}), 401
if api_key != app.config['API_KEY']:
logger.error(f"Invalid API key provided: {api_key}")
return jsonify({"status": "error", "message": "Invalid API key"}), 401
logger.debug("API key validated successfully")
return None
@app.route('/') @app.route('/')
def index(): def index():
return render_template('index.html', site_name='FHIRFLARE IG Toolkit', now=datetime.now()) return render_template('index.html', site_name='FHIRFLARE IG Toolkit', now=datetime.now())
@ -95,20 +110,25 @@ def view_igs():
if os.path.exists(packages_dir): if os.path.exists(packages_dir):
for filename in os.listdir(packages_dir): for filename in os.listdir(packages_dir):
if filename.endswith('.tgz'): if filename.endswith('.tgz'):
name_version = filename[:-4] # Remove .tgz # Split on the last hyphen to separate name and version
parts = name_version.split('-') last_hyphen_index = filename.rfind('-')
version_start = -1 if last_hyphen_index != -1 and filename.endswith('.tgz'):
for i, part in enumerate(parts): name = filename[:last_hyphen_index]
if part[0].isdigit() or part in ('preview', 'current', 'latest'): version = filename[last_hyphen_index + 1:-4] # Remove .tgz
version_start = i # Validate that the version looks reasonable (e.g., starts with a digit or is a known keyword)
break if version[0].isdigit() or version in ('preview', 'current', 'latest'):
if version_start > 0: # Ensure there's a name before version # Replace underscores with dots to match FHIR package naming convention
name = '.'.join(parts[:version_start]) name = name.replace('_', '.')
version = '-'.join(parts[version_start:]) packages.append({'name': name, 'version': version, 'filename': filename})
packages.append({'name': name, 'version': version, 'filename': filename}) else:
# Fallback: treat as name only, log warning
name = filename[:-4]
version = ''
logger.warning(f"Could not parse version from {filename}, treating as name only")
packages.append({'name': name, 'version': version, 'filename': filename})
else: else:
# Fallback: treat as name only, log warning # Fallback: treat as name only, log warning
name = name_version name = filename[:-4]
version = '' version = ''
logger.warning(f"Could not parse version from {filename}, treating as name only") logger.warning(f"Could not parse version from {filename}, treating as name only")
packages.append({'name': name, 'version': version, 'filename': filename}) packages.append({'name': name, 'version': version, 'filename': filename})
@ -141,6 +161,69 @@ def view_igs():
duplicate_groups=duplicate_groups, group_colors=group_colors, duplicate_groups=duplicate_groups, group_colors=group_colors,
site_name='FLARE FHIR IG Toolkit', now=datetime.now()) site_name='FLARE FHIR IG Toolkit', now=datetime.now())
@app.route('/push-igs', methods=['GET', 'POST'])
def push_igs():
igs = ProcessedIg.query.all()
processed_ids = {(ig.package_name, ig.version) for ig in igs}
packages = []
packages_dir = app.config['FHIR_PACKAGES_DIR']
logger.debug(f"Scanning packages directory: {packages_dir}")
if os.path.exists(packages_dir):
for filename in os.listdir(packages_dir):
if filename.endswith('.tgz'):
# Split on the last hyphen to separate name and version
last_hyphen_index = filename.rfind('-')
if last_hyphen_index != -1 and filename.endswith('.tgz'):
name = filename[:last_hyphen_index]
version = filename[last_hyphen_index + 1:-4] # Remove .tgz
# Validate that the version looks reasonable (e.g., starts with a digit or is a known keyword)
if version[0].isdigit() or version in ('preview', 'current', 'latest'):
# Replace underscores with dots to match FHIR package naming convention
name = name.replace('_', '.')
packages.append({'name': name, 'version': version, 'filename': filename})
else:
# Fallback: treat as name only, log warning
name = filename[:-4]
version = ''
logger.warning(f"Could not parse version from {filename}, treating as name only")
packages.append({'name': name, 'version': version, 'filename': filename})
else:
# Fallback: treat as name only, log warning
name = filename[:-4]
version = ''
logger.warning(f"Could not parse version from {filename}, treating as name only")
packages.append({'name': name, 'version': version, 'filename': filename})
logger.debug(f"Found packages: {packages}")
else:
logger.warning(f"Packages directory not found: {packages_dir}")
# Calculate duplicate_names
duplicate_names = {}
for pkg in packages:
name = pkg['name']
if name not in duplicate_names:
duplicate_names[name] = []
duplicate_names[name].append(pkg)
# Calculate duplicate_groups
duplicate_groups = {}
for name, pkgs in duplicate_names.items():
if len(pkgs) > 1: # Only include packages with multiple versions
duplicate_groups[name] = [pkg['version'] for pkg in pkgs]
# Precompute group colors
colors = ['bg-warning', 'bg-info', 'bg-success', 'bg-danger']
group_colors = {}
for i, name in enumerate(duplicate_groups.keys()):
group_colors[name] = colors[i % len(colors)]
return render_template('cp_push_igs.html', packages=packages, processed_list=igs,
processed_ids=processed_ids, duplicate_names=duplicate_names,
duplicate_groups=duplicate_groups, group_colors=group_colors,
site_name='FLARE FHIR IG Toolkit', now=datetime.now(),
api_key=app.config['API_KEY']) # Pass the API key to the template
@app.route('/process-igs', methods=['POST']) @app.route('/process-igs', methods=['POST'])
def process_ig(): def process_ig():
filename = request.form.get('filename') filename = request.form.get('filename')
@ -154,18 +237,15 @@ def process_ig():
return redirect(url_for('view_igs')) return redirect(url_for('view_igs'))
try: try:
name_version = filename[:-4] # Parse name and version from filename
parts = name_version.split('-') last_hyphen_index = filename.rfind('-')
version_start = -1 if last_hyphen_index != -1 and filename.endswith('.tgz'):
for i, part in enumerate(parts): name = filename[:last_hyphen_index]
if part[0].isdigit() or part in ('preview', 'current', 'latest'): version = filename[last_hyphen_index + 1:-4]
version_start = i # Replace underscores with dots to match FHIR package naming convention
break name = name.replace('_', '.')
if version_start > 0:
name = '.'.join(parts[:version_start])
version = '-'.join(parts[version_start:])
else: else:
name = name_version name = filename[:-4]
version = '' version = ''
logger.warning(f"Could not parse version from {filename} during processing") logger.warning(f"Could not parse version from {filename} during processing")
package_info = services.process_package_file(tgz_path) package_info = services.process_package_file(tgz_path)
@ -273,6 +353,227 @@ def get_example_content():
except tarfile.TarError as e: except tarfile.TarError as e:
return jsonify({"error": f"Error reading {tgz_path}: {e}"}), 500 return jsonify({"error": f"Error reading {tgz_path}: {e}"}), 500
# API Endpoint: Import IG Package
@app.route('/api/import-ig', methods=['POST'])
def api_import_ig():
# Check API key
auth_error = check_api_key()
if auth_error:
return auth_error
# Validate request
if not request.is_json:
return jsonify({"status": "error", "message": "Request must be JSON"}), 400
data = request.get_json()
package_name = data.get('package_name')
version = data.get('version')
if not package_name or not version:
return jsonify({"status": "error", "message": "Missing package_name or version"}), 400
# Validate package name and version format using re
if not (isinstance(package_name, str) and isinstance(version, str) and
re.match(r'^[a-zA-Z0-9-]+(\.[a-zA-Z0-9-]+)+$', package_name) and
re.match(r'^[a-zA-Z0-9\.\-]+$', version)):
return jsonify({"status": "error", "message": "Invalid package name or version format"}), 400
try:
# Import package and dependencies
result = services.import_package_and_dependencies(package_name, version)
if result['errors'] and not result['downloaded']:
return jsonify({"status": "error", "message": f"Failed to import {package_name}#{version}: {result['errors'][0]}"}), 500
# Check for duplicates
packages = []
packages_dir = app.config['FHIR_PACKAGES_DIR']
if os.path.exists(packages_dir):
for filename in os.listdir(packages_dir):
if filename.endswith('.tgz'):
# Split on the last hyphen to separate name and version
last_hyphen_index = filename.rfind('-')
if last_hyphen_index != -1 and filename.endswith('.tgz'):
name = filename[:last_hyphen_index]
version = filename[last_hyphen_index + 1:-4]
# Replace underscores with dots to match FHIR package naming convention
name = name.replace('_', '.')
if version[0].isdigit() or version in ('preview', 'current', 'latest'):
packages.append({'name': name, 'version': version, 'filename': filename})
else:
name = filename[:-4]
version = ''
packages.append({'name': name, 'version': version, 'filename': filename})
else:
name = filename[:-4]
version = ''
packages.append({'name': name, 'version': version, 'filename': filename})
# Calculate duplicates
duplicate_names = {}
for pkg in packages:
name = pkg['name']
if name not in duplicate_names:
duplicate_names[name] = []
duplicate_names[name].append(pkg)
duplicates = []
for name, pkgs in duplicate_names.items():
if len(pkgs) > 1:
versions = [pkg['version'] for pkg in pkgs]
duplicates.append(f"{name} (exists as {', '.join(versions)})")
# Deduplicate dependencies
seen = set()
unique_dependencies = []
for dep in result.get('dependencies', []):
dep_str = f"{dep['name']}#{dep['version']}"
if dep_str not in seen:
seen.add(dep_str)
unique_dependencies.append(dep_str)
# Prepare response
response = {
"status": "success",
"message": "Package imported successfully",
"package_name": package_name,
"version": version,
"dependencies": unique_dependencies,
"duplicates": duplicates
}
return jsonify(response), 200
except Exception as e:
logger.error(f"Error in api_import_ig: {str(e)}")
return jsonify({"status": "error", "message": f"Error importing package: {str(e)}"}), 500
# API Endpoint: Push IG to FHIR Server with Streaming
@app.route('/api/push-ig', methods=['POST'])
def api_push_ig():
# Check API key
auth_error = check_api_key()
if auth_error:
return auth_error
# Validate request
if not request.is_json:
return jsonify({"status": "error", "message": "Request must be JSON"}), 400
data = request.get_json()
package_name = data.get('package_name')
version = data.get('version')
fhir_server_url = data.get('fhir_server_url')
include_dependencies = data.get('include_dependencies', True)
if not all([package_name, version, fhir_server_url]):
return jsonify({"status": "error", "message": "Missing package_name, version, or fhir_server_url"}), 400
# Validate package name and version format using re
if not (isinstance(package_name, str) and isinstance(version, str) and
re.match(r'^[a-zA-Z0-9-]+(\.[a-zA-Z0-9-]+)+$', package_name) and
re.match(r'^[a-zA-Z0-9\.\-]+$', version)):
return jsonify({"status": "error", "message": "Invalid package name or version format"}), 400
# Check if package exists
tgz_filename = services._construct_tgz_filename(package_name, version)
tgz_path = os.path.join(app.config['FHIR_PACKAGES_DIR'], tgz_filename)
if not os.path.exists(tgz_path):
return jsonify({"status": "error", "message": f"Package not found: {package_name}#{version}"}), 404
def generate_stream():
try:
# Start message
yield json.dumps({"type": "start", "message": f"Starting push for {package_name}#{version}..."}) + "\n"
# Extract resources from the package
resources = []
with tarfile.open(tgz_path, "r:gz") as tar:
for member in tar.getmembers():
if member.name.startswith('package/') and member.name.endswith('.json'):
with tar.extractfile(member) as f:
resource_data = json.load(f)
if 'resourceType' in resource_data:
resources.append(resource_data)
# If include_dependencies is True, find and include dependencies
pushed_packages = [f"{package_name}#{version}"]
if include_dependencies:
yield json.dumps({"type": "progress", "message": "Processing dependencies..."}) + "\n"
# Re-import to get dependencies (simulating dependency resolution)
import_result = services.import_package_and_dependencies(package_name, version)
dependencies = import_result.get('dependencies', [])
for dep in dependencies:
dep_name = dep['name']
dep_version = dep['version']
dep_tgz_filename = services._construct_tgz_filename(dep_name, dep_version)
dep_tgz_path = os.path.join(app.config['FHIR_PACKAGES_DIR'], dep_tgz_filename)
if os.path.exists(dep_tgz_path):
with tarfile.open(dep_tgz_path, "r:gz") as tar:
for member in tar.getmembers():
if member.name.startswith('package/') and member.name.endswith('.json'):
with tar.extractfile(member) as f:
resource_data = json.load(f)
if 'resourceType' in resource_data:
resources.append(resource_data)
pushed_packages.append(f"{dep_name}#{dep_version}")
yield json.dumps({"type": "progress", "message": f"Added dependency {dep_name}#{dep_version}"}) + "\n"
else:
yield json.dumps({"type": "warning", "message": f"Dependency {dep_name}#{dep_version} not found, skipping"}) + "\n"
# Push resources to FHIR server
server_response = []
success_count = 0
failure_count = 0
total_resources = len(resources)
yield json.dumps({"type": "progress", "message": f"Found {total_resources} resources to upload"}) + "\n"
for i, resource in enumerate(resources, 1):
resource_type = resource.get('resourceType')
resource_id = resource.get('id')
if not resource_type or not resource_id:
yield json.dumps({"type": "warning", "message": f"Skipping invalid resource at index {i}"}) + "\n"
failure_count += 1
continue
# Construct the FHIR server URL for the resource
resource_url = f"{fhir_server_url.rstrip('/')}/{resource_type}/{resource_id}"
yield json.dumps({"type": "progress", "message": f"Uploading {resource_type}/{resource_id} ({i}/{total_resources})..."}) + "\n"
try:
response = requests.put(resource_url, json=resource, headers={'Content-Type': 'application/fhir+json'})
response.raise_for_status()
server_response.append(f"Uploaded {resource_type}/{resource_id} successfully")
yield json.dumps({"type": "success", "message": f"Uploaded {resource_type}/{resource_id} successfully"}) + "\n"
success_count += 1
except requests.exceptions.RequestException as e:
error_msg = f"Failed to upload {resource_type}/{resource_id}: {str(e)}"
server_response.append(error_msg)
yield json.dumps({"type": "error", "message": error_msg}) + "\n"
failure_count += 1
# Final summary
summary = {
"status": "success" if failure_count == 0 else "partial",
"message": f"Push completed: {success_count} resources uploaded, {failure_count} failed",
"package_name": package_name,
"version": version,
"pushed_packages": pushed_packages,
"server_response": "; ".join(server_response) if server_response else "No resources uploaded",
"success_count": success_count,
"failure_count": failure_count
}
yield json.dumps({"type": "complete", "data": summary}) + "\n"
except Exception as e:
logger.error(f"Error in api_push_ig: {str(e)}")
error_response = {
"status": "error",
"message": f"Error pushing package: {str(e)}"
}
yield json.dumps({"type": "error", "message": error_response["message"]}) + "\n"
yield json.dumps({"type": "complete", "data": error_response}) + "\n"
return Response(generate_stream(), mimetype='application/x-ndjson')
with app.app_context(): with app.app_context():
logger.debug(f"Creating database at: {app.config['SQLALCHEMY_DATABASE_URI']}") logger.debug(f"Creating database at: {app.config['SQLALCHEMY_DATABASE_URI']}")
db.create_all() db.create_all()

Binary file not shown.

View File

@ -3,4 +3,5 @@ Flask-SQLAlchemy==3.0.5
Werkzeug==2.3.7 Werkzeug==2.3.7
requests==2.31.0 requests==2.31.0
Flask-WTF==1.2.1 Flask-WTF==1.2.1
WTForms==3.1.2 WTForms==3.1.2
Pytest

View File

@ -189,7 +189,14 @@ def import_package_and_dependencies(initial_name, initial_version):
# --- FIX: Ensure consistent indentation --- # --- FIX: Ensure consistent indentation ---
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
logger.info(f"Starting recursive import for {initial_name}#{initial_version}") logger.info(f"Starting recursive import for {initial_name}#{initial_version}")
results = {'requested': (initial_name, initial_version), 'processed': set(), 'downloaded': {}, 'all_dependencies': {}, 'errors': [] } results = {
'requested': (initial_name, initial_version),
'processed': set(),
'downloaded': {},
'all_dependencies': {},
'dependencies': [], # New field to store dependencies as a list
'errors': []
}
pending_queue = [(initial_name, initial_version)] pending_queue = [(initial_name, initial_version)]
processed_lookup = set() processed_lookup = set()
@ -223,15 +230,17 @@ def import_package_and_dependencies(initial_name, initial_version):
results['all_dependencies'][package_id_tuple] = dependencies results['all_dependencies'][package_id_tuple] = dependencies
results['processed'].add(package_id_tuple) results['processed'].add(package_id_tuple)
logger.debug(f"Dependencies for {name}#{version}: {list(dependencies.keys())}") logger.debug(f"Dependencies for {name}#{version}: {list(dependencies.keys())}")
# Add dependencies to the new 'dependencies' list
for dep_name, dep_version in dependencies.items(): for dep_name, dep_version in dependencies.items():
if isinstance(dep_name, str) and isinstance(dep_version, str) and dep_name and dep_version: if isinstance(dep_name, str) and isinstance(dep_version, str) and dep_name and dep_version:
dep_tuple = (dep_name, dep_version) dep_tuple = (dep_name, dep_version)
if dep_tuple not in processed_lookup: results['dependencies'].append({"name": dep_name, "version": dep_version})
if dep_tuple not in pending_queue: if dep_tuple not in processed_lookup:
pending_queue.append(dep_tuple) if dep_tuple not in pending_queue:
logger.debug(f"Added to queue: {dep_name}#{dep_version}") pending_queue.append(dep_tuple)
else: logger.debug(f"Added to queue: {dep_name}#{dep_version}")
logger.warning(f"Skipping invalid dependency '{dep_name}': '{dep_version}' in {name}#{version}") else:
logger.warning(f"Skipping invalid dependency '{dep_name}': '{dep_version}' in {name}#{version}")
# --- End Correctly indented block --- # --- End Correctly indented block ---
proc_count=len(results['processed']); dl_count=len(results['downloaded']); err_count=len(results['errors']) proc_count=len(results['processed']); dl_count=len(results['downloaded']); err_count=len(results['errors'])

View File

@ -26,6 +26,9 @@
<li class="nav-item"> <li class="nav-item">
<a class="nav-link {{ 'active' if request.endpoint == 'import_ig' else '' }}" href="{{ url_for('import_ig') }}"><i class="bi bi-download me-1"></i> Import IGs</a> <a class="nav-link {{ 'active' if request.endpoint == 'import_ig' else '' }}" href="{{ url_for('import_ig') }}"><i class="bi bi-download me-1"></i> Import IGs</a>
</li> </li>
<li class="nav-item">
<a class="nav-link {{ 'active' if request.endpoint == 'push_igs' else '' }}" href="{{ url_for('push_igs') }}"><i class="bi bi-upload me-1"></i> Push IGs</a>
</li>
</ul> </ul>
</div> </div>
</div> </div>

View File

@ -11,6 +11,7 @@
<div class="d-grid gap-2 d-sm-flex justify-content-sm-center"> <div class="d-grid gap-2 d-sm-flex justify-content-sm-center">
<a href="{{ url_for('import_ig') }}" class="btn btn-primary btn-lg px-4 gap-3">Import IGs</a> <a href="{{ url_for('import_ig') }}" class="btn btn-primary btn-lg px-4 gap-3">Import IGs</a>
<a href="{{ url_for('view_igs') }}" class="btn btn-outline-secondary btn-lg px-4 disabled">Manage FHIR Packages</a> <a href="{{ url_for('view_igs') }}" class="btn btn-outline-secondary btn-lg px-4 disabled">Manage FHIR Packages</a>
<a href="{{ url_for('push_igs') }}" class="btn btn-outline-secondary btn-lg px-4 ">Upload IG's</a>
</div> </div>
</div> </div>
</div> </div>

328
templates/cp_push_igs.html Normal file
View File

@ -0,0 +1,328 @@
{% extends "base.html" %}
{% block content %}
<div class="container-fluid">
<div class="row">
<!-- Left Column: List of Downloaded IGs -->
<div class="col-md-6">
<h2>Downloaded IGs</h2>
{% if packages %}
<table class="table table-striped">
<thead>
<tr>
<th>Package Name</th>
<th>Version</th>
<th>Actions</th>
</tr>
</thead>
<tbody>
{% for pkg in packages %}
{% set name = pkg.name %}
{% set version = pkg.version %}
{% set filename = pkg.filename %}
{% set processed = (name, version) in processed_ids %}
{% set duplicate_group = duplicate_groups.get(name) %}
<tr {% if duplicate_group %}class="{{ group_colors[name] }}"{% endif %}>
<td>{{ name }}</td>
<td>{{ version }}</td>
<td>
{% if not processed %}
<form method="POST" action="{{ url_for('process_ig') }}" style="display:inline;">
<input type="hidden" name="filename" value="{{ filename }}">
<button type="submit" class="btn btn-primary btn-sm">Process</button>
</form>
{% endif %}
<form method="POST" action="{{ url_for('delete_ig') }}" style="display:inline;" onsubmit="return confirm('Are you sure you want to delete {{ name }}#{{ version }}?');">
<input type="hidden" name="filename" value="{{ filename }}">
<button type="submit" class="btn btn-danger btn-sm">Delete</button>
</form>
</td>
</tr>
{% endfor %}
</tbody>
</table>
{% if duplicate_groups %}
<div class="alert alert-warning">
<strong>Duplicate Packages Detected:</strong>
<ul>
{% for name, versions in duplicate_groups.items() %}
<li>{{ name }} (Versions: {{ versions | join(', ') }})</li>
{% endfor %}
</ul>
<p>Please resolve duplicates by deleting older versions to avoid conflicts.</p>
</div>
{% endif %}
{% else %}
<p>No packages downloaded yet. Use the "Import IG" tab to download packages.</p>
{% endif %}
</div>
<!-- Right Column: Push IGs Form -->
<div class="col-md-6">
<h2>Push IGs to FHIR Server</h2>
<form id="pushIgForm" method="POST">
<div class="mb-3">
<label for="packageSelect" class="form-label">Select Package to Push</label>
<select class="form-select" id="packageSelect" name="package_id" required>
<option value="" disabled selected>Select a package...</option>
{% for pkg in packages %}
<option value="{{ pkg.name }}#{{ pkg.version }}">{{ pkg.name }}#{{ pkg.version }}</option>
{% endfor %}
</select>
</div>
<div class="mb-3">
<label for="fhirServerUrl" class="form-label">FHIR Server URL</label>
<input type="url" class="form-control" id="fhirServerUrl" name="fhir_server_url" placeholder="e.g., http://hapi.fhir.org/baseR4" required>
</div>
<div class="mb-3 form-check">
<input type="checkbox" class="form-check-input" id="includeDependencies" name="include_dependencies" checked>
<label class="form-check-label" for="includeDependencies">Include Dependencies</label>
</div>
<button type="submit" class="btn btn-primary" id="pushButton">Push to FHIR Server</button>
</form>
<!-- Live Console -->
<div class="mt-3">
<h4>Live Console</h4>
<div id="liveConsole" class="border p-3 bg-dark text-light" style="height: 300px; overflow-y: auto; font-family: monospace; font-size: 14px;">
<div>Console output will appear here...</div>
</div>
</div>
<!-- Response Area -->
<div id="pushResponse" class="mt-3">
{% with messages = get_flashed_messages(with_categories=true) %}
{% if messages %}
{% for category, message in messages %}
<div class="alert alert-{{ category }}"> {{ message }}</div>
{% endfor %}
{% endif %}
{% endwith %}
</div>
</div>
</div>
</div>
<script>
document.getElementById('pushIgForm').addEventListener('submit', async function(event) {
event.preventDefault();
const form = event.target;
const pushButton = document.getElementById('pushButton');
const formData = new FormData(form);
const packageId = formData.get('package_id');
const [packageName, version] = packageId.split('#');
const fhirServerUrl = formData.get('fhir_server_url');
const includeDependencies = formData.get('include_dependencies') === 'on';
// Disable the button to prevent multiple submissions
pushButton.disabled = true;
pushButton.textContent = 'Pushing...';
// Clear the console and response area
const liveConsole = document.getElementById('liveConsole');
const responseDiv = document.getElementById('pushResponse');
liveConsole.innerHTML = '<div>Starting push operation...</div>';
responseDiv.innerHTML = '';
try {
const response = await fetch('/api/push-ig', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Accept': 'application/x-ndjson'
},
body: JSON.stringify({
package_name: packageName,
version: version,
fhir_server_url: fhirServerUrl,
include_dependencies: includeDependencies,
api_key: '{{ api_key | safe }}' // Use the API key passed from the server
})
});
if (!response.ok) {
const errorData = await response.json();
throw new Error(errorData.message || 'Failed to push package');
}
// Stream the response
const reader = response.body.getReader();
const decoder = new TextDecoder();
let buffer = '';
while (true) {
const { done, value } = await reader.read();
if (done) break;
// Decode the chunk and append to buffer
buffer += decoder.decode(value, { stream: true });
// Process each line (newline-separated JSON)
const lines = buffer.split('\n');
buffer = lines.pop(); // Keep the last (possibly incomplete) line in the buffer
for (const line of lines) {
if (!line.trim()) continue; // Skip empty lines
try {
const data = JSON.parse(line);
const timestamp = new Date().toLocaleTimeString();
let messageClass = '';
let prefix = '';
switch (data.type) {
case 'start':
case 'progress':
messageClass = 'text-info';
prefix = '[INFO]';
break;
case 'success':
messageClass = 'text-success';
prefix = '[SUCCESS]';
break;
case 'error':
messageClass = 'text-danger';
prefix = '[ERROR]';
break;
case 'warning':
messageClass = 'text-warning';
prefix = '[WARNING]';
break;
case 'complete':
// Display the final summary in the response area
if (data.data.status === 'success') {
responseDiv.innerHTML = `
<div class="alert alert-success">
<strong>Success:</strong> ${data.data.message}<br>
<strong>Pushed Packages:</strong> ${data.data.pushed_packages.join(', ')}<br>
<strong>Server Response:</strong> ${data.data.server_response}
</div>
`;
} else if (data.data.status === 'partial') {
responseDiv.innerHTML = `
<div class="alert alert-warning">
<strong>Partial Success:</strong> ${data.data.message}<br>
<strong>Pushed Packages:</strong> ${data.data.pushed_packages.join(', ')}<br>
<strong>Server Response:</strong> ${data.data.server_response}
</div>
`;
} else {
responseDiv.innerHTML = `
<div class="alert alert-danger">
<strong>Error:</strong> ${data.data.message}
</div>
`;
}
// Also log the completion in the console
messageClass = data.data.status === 'success' ? 'text-success' : (data.data.status === 'partial' ? 'text-warning' : 'text-danger');
prefix = data.data.status === 'success' ? '[SUCCESS]' : (data.data.status === 'partial' ? '[PARTIAL]' : '[ERROR]');
break;
default:
messageClass = 'text-light';
prefix = '[INFO]';
}
const messageDiv = document.createElement('div');
messageDiv.className = messageClass;
messageDiv.textContent = `${timestamp} ${prefix} ${data.message || (data.data && data.data.message) || 'Unknown message'}`;
liveConsole.appendChild(messageDiv);
// Auto-scroll to the bottom
liveConsole.scrollTop = liveConsole.scrollHeight;
} catch (parseError) {
console.error('Error parsing streamed message:', parseError, 'Line:', line);
}
}
}
// Process any remaining buffer
if (buffer.trim()) {
try {
const data = JSON.parse(buffer);
const timestamp = new Date().toLocaleTimeString();
let messageClass = '';
let prefix = '';
switch (data.type) {
case 'start':
case 'progress':
messageClass = 'text-info';
prefix = '[INFO]';
break;
case 'success':
messageClass = 'text-success';
prefix = '[SUCCESS]';
break;
case 'error':
messageClass = 'text-danger';
prefix = '[ERROR]';
break;
case 'warning':
messageClass = 'text-warning';
prefix = '[WARNING]';
break;
case 'complete':
if (data.data.status === 'success') {
responseDiv.innerHTML = `
<div class="alert alert-success">
<strong>Success:</strong> ${data.data.message}<br>
<strong>Pushed Packages:</strong> ${data.data.pushed_packages.join(', ')}<br>
<strong>Server Response:</strong> ${data.data.server_response}
</div>
`;
} else if (data.data.status === 'partial') {
responseDiv.innerHTML = `
<div class="alert alert-warning">
<strong>Partial Success:</strong> ${data.data.message}<br>
<strong>Pushed Packages:</strong> ${data.data.pushed_packages.join(', ')}<br>
<strong>Server Response:</strong> ${data.data.server_response}
</div>
`;
} else {
responseDiv.innerHTML = `
<div class="alert alert-danger">
<strong>Error:</strong> ${data.data.message}
</div>
`;
}
messageClass = data.data.status === 'success' ? 'text-success' : (data.data.status === 'partial' ? 'text-warning' : 'text-danger');
prefix = data.data.status === 'success' ? '[SUCCESS]' : (data.data.status === 'partial' ? '[PARTIAL]' : '[ERROR]');
break;
default:
messageClass = 'text-light';
prefix = '[INFO]';
}
const messageDiv = document.createElement('div');
messageDiv.className = messageClass;
messageDiv.textContent = `${timestamp} ${prefix} ${data.message || (data.data && data.data.message) || 'Unknown message'}`;
liveConsole.appendChild(messageDiv);
// Auto-scroll to the bottom
liveConsole.scrollTop = liveConsole.scrollHeight;
} catch (parseError) {
console.error('Error parsing final streamed message:', parseError, 'Buffer:', buffer);
}
}
} catch (error) {
const timestamp = new Date().toLocaleTimeString();
const errorDiv = document.createElement('div');
errorDiv.className = 'text-danger';
errorDiv.textContent = `${timestamp} [ERROR] Failed to push package: ${error.message}`;
liveConsole.appendChild(errorDiv);
liveConsole.scrollTop = liveConsole.scrollHeight;
responseDiv.innerHTML = `
<div class="alert alert-danger">
<strong>Error:</strong> Failed to push package: ${error.message}
</div>
`;
} finally {
// Re-enable the button
pushButton.disabled = false;
pushButton.textContent = 'Push to FHIR Server';
}
});
</script>
{% endblock %}

View File

@ -13,6 +13,7 @@ center">
<div class="d-grid gap-2 d-sm-flex justify-content-sm-center"> <div class="d-grid gap-2 d-sm-flex justify-content-sm-center">
<a href="{{ url_for('index') }}" class="btn btn-primary btn-lg px-4 gap-3">Back to Home</a> <a href="{{ url_for('index') }}" class="btn btn-primary btn-lg px-4 gap-3">Back to Home</a>
<a href="{{ url_for('view_igs') }}" class="btn btn-outline-secondary btn-lg px-4">Manage FHIR Packages</a> <a href="{{ url_for('view_igs') }}" class="btn btn-outline-secondary btn-lg px-4">Manage FHIR Packages</a>
<a href="{{ url_for('push_igs') }}" class="btn btn-outline-secondary btn-lg px-4">Upload IG's</a>
</div> </div>
</div> </div>
</div> </div>

View File

@ -13,6 +13,7 @@
<div class="d-grid gap-2 d-sm-flex justify-content-sm-center"> <div class="d-grid gap-2 d-sm-flex justify-content-sm-center">
<a href="{{ url_for('index') }}" class="btn btn-primary btn-lg px-4 gap-3">Back to Home</a> <a href="{{ url_for('index') }}" class="btn btn-primary btn-lg px-4 gap-3">Back to Home</a>
<a href="{{ url_for('view_igs') }}" class="btn btn-outline-secondary btn-lg px-4">View Downloaded IGs</a> <a href="{{ url_for('view_igs') }}" class="btn btn-outline-secondary btn-lg px-4">View Downloaded IGs</a>
<a href="{{ url_for('push_igs') }}" class="btn btn-outline-secondary btn-lg px-4">Upload IG's</a>
</div> </div>
</div> </div>
</div> </div>

View File

@ -11,6 +11,8 @@
<div class="d-grid gap-2 d-sm-flex justify-content-sm-center"> <div class="d-grid gap-2 d-sm-flex justify-content-sm-center">
<a href="{{ url_for('import_ig') }}" class="btn btn-primary btn-lg px-4 gap-3">Import FHIR IG</a> <a href="{{ url_for('import_ig') }}" class="btn btn-primary btn-lg px-4 gap-3">Import FHIR IG</a>
<a href="{{ url_for('view_igs') }}" class="btn btn-outline-secondary btn-lg px-4">View Downloaded IGs</a> <a href="{{ url_for('view_igs') }}" class="btn btn-outline-secondary btn-lg px-4">View Downloaded IGs</a>
<a href="{{ url_for('push_igs') }}" class="btn btn-outline-secondary btn-lg px-4">Upload IG's</a>
</div> </div>
</div> </div>
</div> </div>

420
tests/test_app.py Normal file
View File

@ -0,0 +1,420 @@
import unittest
import pytest
import os
import sys
import json
import tarfile
import shutil
from unittest.mock import patch, MagicMock
from flask import Flask, url_for
from flask.testing import FlaskClient
# Add the parent directory (/app) to sys.path
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
from app import app, db, ProcessedIg
from datetime import datetime
class TestFHIRFlareIGToolkit(unittest.TestCase):
def setUp(self):
# Configure the Flask app for testing
app.config['TESTING'] = True
app.config['WTF_CSRF_ENABLED'] = False # Disable CSRF for testing
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///:memory:'
app.config['FHIR_PACKAGES_DIR'] = 'test_packages'
app.config['SECRET_KEY'] = 'test-secret-key'
app.config['API_KEY'] = 'test-api-key'
# Create the test packages directory
os.makedirs(app.config['FHIR_PACKAGES_DIR'], exist_ok=True)
# Create the Flask test client
self.client = app.test_client()
# Initialize the database
with app.app_context():
db.create_all()
def tearDown(self):
# Clean up the database and test packages directory
with app.app_context():
db.session.remove()
db.drop_all()
if os.path.exists(app.config['FHIR_PACKAGES_DIR']):
shutil.rmtree(app.config['FHIR_PACKAGES_DIR'])
# Helper method to create a mock .tgz file
def create_mock_tgz(self, filename, content=None):
tgz_path = os.path.join(app.config['FHIR_PACKAGES_DIR'], filename)
with tarfile.open(tgz_path, "w:gz") as tar:
if content:
# Create a mock package.json file inside the .tgz
import io
package_json = io.BytesIO(json.dumps(content).encode('utf-8'))
tarinfo = tarfile.TarInfo(name="package/package.json")
tarinfo.size = len(package_json.getvalue())
tar.addfile(tarinfo, package_json)
return tgz_path
# Test Case 1: Test Homepage Rendering
def test_homepage(self):
response = self.client.get('/')
self.assertEqual(response.status_code, 200)
self.assertIn(b'FHIRFLARE IG Toolkit', response.data)
# Test Case 2: Test Import IG Page Rendering
def test_import_ig_page(self):
response = self.client.get('/import-ig')
self.assertEqual(response.status_code, 200)
self.assertIn(b'Import IG', response.data)
self.assertIn(b'Package Name', response.data)
self.assertIn(b'Package Version', response.data)
# Test Case 3: Test Import IG Form Submission (Success)
@patch('services.import_package_and_dependencies')
def test_import_ig_success(self, mock_import):
mock_import.return_value = {
'downloaded': True,
'errors': [],
'dependencies': [{'name': 'hl7.fhir.r4.core', 'version': '4.0.1'}]
}
response = self.client.post('/import-ig', data={
'package_name': 'hl7.fhir.us.core',
'package_version': '3.1.1'
}, follow_redirects=True)
self.assertEqual(response.status_code, 200)
self.assertIn(b'Successfully downloaded hl7.fhir.us.core#3.1.1', response.data)
# Test Case 4: Test Import IG Form Submission (Failure)
@patch('services.import_package_and_dependencies')
def test_import_ig_failure(self, mock_import):
mock_import.return_value = {
'downloaded': False,
'errors': ['Package not found'],
'dependencies': []
}
response = self.client.post('/import-ig', data={
'package_name': 'invalid.package',
'package_version': '1.0.0'
}, follow_redirects=True)
self.assertEqual(response.status_code, 200)
self.assertIn(b'Failed to import invalid.package#1.0.0', response.data)
# Test Case 5: Test Import IG Form Submission (Invalid Input)
def test_import_ig_invalid_input(self):
response = self.client.post('/import-ig', data={
'package_name': 'invalid@package', # Invalid format
'package_version': '1.0.0'
}, follow_redirects=True)
self.assertEqual(response.status_code, 200)
self.assertIn(b'Invalid package name format', response.data)
# Test Case 6: Test View IGs Page Rendering (No Packages)
def test_view_igs_no_packages(self):
response = self.client.get('/view-igs')
self.assertEqual(response.status_code, 200)
self.assertIn(b'No packages downloaded yet', response.data)
# Test Case 7: Test View IGs Page Rendering (With Packages)
def test_view_igs_with_packages(self):
# Create a mock .tgz file
self.create_mock_tgz('hl7.fhir.us.core-3.1.1.tgz', {
'name': 'hl7.fhir.us.core',
'version': '3.1.1'
})
response = self.client.get('/view-igs')
self.assertEqual(response.status_code, 200)
self.assertIn(b'hl7.fhir.us.core', response.data)
self.assertIn(b'3.1.1', response.data)
# Test Case 8: Test Process IG (Success)
@patch('services.process_package_file')
def test_process_ig_success(self, mock_process):
mock_process.return_value = {
'resource_types_info': [{'type': 'Patient'}],
'must_support_elements': {'Patient': ['name']},
'examples': {'Patient': ['example1.json']}
}
self.create_mock_tgz('hl7.fhir.us.core-3.1.1.tgz')
response = self.client.post('/process-igs', data={
'filename': 'hl7.fhir.us.core-3.1.1.tgz'
}, follow_redirects=True)
self.assertEqual(response.status_code, 200)
self.assertIn(b'Successfully processed hl7.fhir.us.core#3.1.1', response.data)
# Test Case 9: Test Process IG (Invalid File)
def test_process_ig_invalid_file(self):
response = self.client.post('/process-igs', data={
'filename': 'invalid-file.txt'
}, follow_redirects=True)
self.assertEqual(response.status_code, 200)
self.assertIn(b'Invalid package file', response.data)
# Test Case 10: Test Delete IG (Success)
def test_delete_ig_success(self):
self.create_mock_tgz('hl7.fhir.us.core-3.1.1.tgz')
response = self.client.post('/delete-ig', data={
'filename': 'hl7.fhir.us.core-3.1.1.tgz'
}, follow_redirects=True)
self.assertEqual(response.status_code, 200)
self.assertIn(b'Deleted hl7.fhir.us.core-3.1.1.tgz', response.data)
self.assertFalse(os.path.exists(os.path.join(app.config['FHIR_PACKAGES_DIR'], 'hl7.fhir.us.core-3.1.1.tgz')))
# Test Case 11: Test Delete IG (File Not Found)
def test_delete_ig_file_not_found(self):
response = self.client.post('/delete-ig', data={
'filename': 'nonexistent.tgz'
}, follow_redirects=True)
self.assertEqual(response.status_code, 200)
self.assertIn(b'File not found: nonexistent.tgz', response.data)
# Test Case 12: Test Unload IG (Success)
def test_unload_ig_success(self):
with app.app_context():
processed_ig = ProcessedIg(
package_name='hl7.fhir.us.core',
version='3.1.1',
processed_date=datetime.now(),
resource_types_info=[{'type': 'Patient'}]
)
db.session.add(processed_ig)
db.session.commit()
ig_id = processed_ig.id
response = self.client.post('/unload-ig', data={
'ig_id': ig_id
}, follow_redirects=True)
self.assertEqual(response.status_code, 200)
self.assertIn(b'Unloaded hl7.fhir.us.core#3.1.1', response.data)
# Test Case 13: Test Unload IG (Invalid ID)
def test_unload_ig_invalid_id(self):
response = self.client.post('/unload-ig', data={
'ig_id': '999'
}, follow_redirects=True)
self.assertEqual(response.status_code, 200)
self.assertIn(b'Package not found with ID: 999', response.data)
# Test Case 14: Test View Processed IG Page
def test_view_processed_ig(self):
with app.app_context():
processed_ig = ProcessedIg(
package_name='hl7.fhir.us.core',
version='3.1.1',
processed_date=datetime.now(),
resource_types_info=[{'type': 'Patient', 'is_profile': False}]
)
db.session.add(processed_ig)
db.session.commit()
ig_id = processed_ig.id
response = self.client.get(f'/view-ig/{ig_id}')
self.assertEqual(response.status_code, 200)
self.assertIn(b'View hl7.fhir.us.core#3.1.1', response.data)
# Test Case 15: Test Push IGs Page Rendering
def test_push_igs_page(self):
response = self.client.get('/push-igs')
self.assertEqual(response.status_code, 200)
self.assertIn(b'Push IGs to FHIR Server', response.data)
self.assertIn(b'Live Console', response.data)
# Test Case 16: Test API - Import IG (Success)
@patch('services.import_package_and_dependencies')
def test_api_import_ig_success(self, mock_import):
mock_import.return_value = {
'downloaded': True,
'errors': [],
'dependencies': [{'name': 'hl7.fhir.r4.core', 'version': '4.0.1'}]
}
response = self.client.post('/api/import-ig',
data=json.dumps({
'package_name': 'hl7.fhir.us.core',
'version': '3.1.1',
'api_key': 'test-api-key'
}),
content_type='application/json'
)
self.assertEqual(response.status_code, 200)
data = json.loads(response.data)
self.assertEqual(data['status'], 'success')
self.assertEqual(data['package_name'], 'hl7.fhir.us.core')
# Test Case 17: Test API - Import IG (Invalid API Key)
def test_api_import_ig_invalid_api_key(self):
response = self.client.post('/api/import-ig',
data=json.dumps({
'package_name': 'hl7.fhir.us.core',
'version': '3.1.1',
'api_key': 'wrong-api-key'
}),
content_type='application/json'
)
self.assertEqual(response.status_code, 401)
data = json.loads(response.data)
self.assertEqual(data['message'], 'Invalid API key')
# Test Case 18: Test API - Import IG (Missing Parameters)
def test_api_import_ig_missing_params(self):
response = self.client.post('/api/import-ig',
data=json.dumps({
'package_name': 'hl7.fhir.us.core',
'api_key': 'test-api-key'
}),
content_type='application/json'
)
self.assertEqual(response.status_code, 400)
data = json.loads(response.data)
self.assertEqual(data['message'], 'Missing package_name or version')
# Test Case 19: Test API - Push IG (Success)
@patch('requests.put')
def test_api_push_ig_success(self, mock_put):
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.raise_for_status.return_value = None
mock_put.return_value = mock_response
self.create_mock_tgz('hl7.fhir.us.core-3.1.1.tgz', {
'name': 'hl7.fhir.us.core',
'version': '3.1.1'
})
# Add a mock resource file
with tarfile.open(os.path.join(app.config['FHIR_PACKAGES_DIR'], 'hl7.fhir.us.core-3.1.1.tgz'), "a:gz") as tar:
resource_data = json.dumps({
'resourceType': 'Patient',
'id': 'example'
}).encode('utf-8')
import io
resource_file = io.BytesIO(resource_data)
tarinfo = tarfile.TarInfo(name="package/Patient-example.json")
tarinfo.size = len(resource_data)
tar.addfile(tarinfo, resource_file)
response = self.client.post('/api/push-ig',
data=json.dumps({
'package_name': 'hl7.fhir.us.core',
'version': '3.1.1',
'fhir_server_url': 'http://test-server/fhir',
'include_dependencies': False,
'api_key': 'test-api-key'
}),
content_type='application/json',
headers={'Accept': 'application/x-ndjson'}
)
self.assertEqual(response.status_code, 200)
response_text = response.data.decode('utf-8')
self.assertIn('"type": "start"', response_text)
self.assertIn('"type": "success"', response_text)
self.assertIn('"status": "success"', response_text)
# Test Case 20: Test API - Push IG (Invalid API Key)
def test_api_push_ig_invalid_api_key(self):
response = self.client.post('/api/push-ig',
data=json.dumps({
'package_name': 'hl7.fhir.us.core',
'version': '3.1.1',
'fhir_server_url': 'http://test-server/fhir',
'include_dependencies': False,
'api_key': 'wrong-api-key'
}),
content_type='application/json',
headers={'Accept': 'application/x-ndjson'}
)
self.assertEqual(response.status_code, 401)
data = json.loads(response.data)
self.assertEqual(data['message'], 'Invalid API key')
# Test Case 21: Test API - Push IG (Package Not Found)
def test_api_push_ig_package_not_found(self):
response = self.client.post('/api/push-ig',
data=json.dumps({
'package_name': 'hl7.fhir.us.core',
'version': '3.1.1',
'fhir_server_url': 'http://test-server/fhir',
'include_dependencies': False,
'api_key': 'test-api-key'
}),
content_type='application/json',
headers={'Accept': 'application/x-ndjson'}
)
self.assertEqual(response.status_code, 404)
data = json.loads(response.data)
self.assertEqual(data['message'], 'Package not found: hl7.fhir.us.core#3.1.1')
# Test Case 22: Test Secret Key - CSRF Protection
def test_secret_key_csrf(self):
# Re-enable CSRF for this test
app.config['WTF_CSRF_ENABLED'] = True
response = self.client.post('/import-ig', data={
'package_name': 'hl7.fhir.us.core',
'package_version': '3.1.1'
}, follow_redirects=True)
self.assertEqual(response.status_code, 400) # CSRF token missing
# Test Case 23: Test Secret Key - Flash Messages
def test_secret_key_flash_messages(self):
# Set a flash message
with self.client as client:
with client.session_transaction() as sess:
sess['_flashes'] = [('success', 'Test message')]
response = self.client.get('/push-igs')
self.assertEqual(response.status_code, 200)
self.assertIn(b'Test message', response.data)
# Test Case 24: Test Get Structure Definition (Success)
def test_get_structure_definition_success(self):
self.create_mock_tgz('hl7.fhir.us.core-3.1.1.tgz')
with tarfile.open(os.path.join(app.config['FHIR_PACKAGES_DIR'], 'hl7.fhir.us.core-3.1.1.tgz'), "a:gz") as tar:
sd_data = json.dumps({
'snapshot': {'element': [{'id': 'Patient.name'}]}
}).encode('utf-8')
import io
sd_file = io.BytesIO(sd_data)
tarinfo = tarfile.TarInfo(name="package/StructureDefinition-Patient.json")
tarinfo.size = len(sd_data)
tar.addfile(tarinfo, sd_file)
response = self.client.get('/get-structure?package_name=hl7.fhir.us.core&package_version=3.1.1&resource_type=Patient')
self.assertEqual(response.status_code, 200)
data = json.loads(response.data)
self.assertIn('elements', data)
self.assertEqual(data['elements'][0]['id'], 'Patient.name')
# Test Case 25: Test Get Structure Definition (Not Found)
def test_get_structure_definition_not_found(self):
self.create_mock_tgz('hl7.fhir.us.core-3.1.1.tgz')
response = self.client.get('/get-structure?package_name=hl7.fhir.us.core&package_version=3.1.1&resource_type=Observation')
self.assertEqual(response.status_code, 404)
data = json.loads(response.data)
self.assertEqual(data['error'], "SD for 'Observation' not found.")
# Test Case 26: Test Get Example Content (Success)
def test_get_example_content_success(self):
self.create_mock_tgz('hl7.fhir.us.core-3.1.1.tgz')
with tarfile.open(os.path.join(app.config['FHIR_PACKAGES_DIR'], 'hl7.fhir.us.core-3.1.1.tgz'), "a:gz") as tar:
example_data = json.dumps({
'resourceType': 'Patient',
'id': 'example'
}).encode('utf-8')
import io
example_file = io.BytesIO(example_data)
tarinfo = tarfile.TarInfo(name="package/example-Patient.json")
tarinfo.size = len(example_data)
tar.addfile(tarinfo, example_file)
response = self.client.get('/get-example?package_name=hl7.fhir.us.core&package_version=3.1.1&filename=package/example-Patient.json')
self.assertEqual(response.status_code, 200)
data = json.loads(response.data)
self.assertEqual(data['resourceType'], 'Patient')
# Test Case 27: Test Get Example Content (Invalid Path)
def test_get_example_content_invalid_path(self):
self.create_mock_tgz('hl7.fhir.us.core-3.1.1.tgz')
response = self.client.get('/get-example?package_name=hl7.fhir.us.core&package_version=3.1.1&filename=invalid/example.json')
self.assertEqual(response.status_code, 400)
data = json.loads(response.data)
self.assertEqual(data['error'], 'Invalid example file path.')
if __name__ == '__main__':
unittest.main()