symbolix-be-py_kit

Python Services Development Kit

Conda Setup

To set up your development environment, create a new Conda environment with Python 3.12.3:

conda create --prefix .venv/ python=3.12.3

After creating the environment, make sure to activate it before proceeding with the next steps:

conda activate .venv/

Development Environment

Port Range

• Core Dependencies: 55581-55599 Ports reserved for essential services such as databases and caches.
• Frontend Services: 55561-55580 Ports allocated for frontend applications or services.
• Backend Services: 55501-55560 Ports used by backend microservices.

Shared Docker Network

Create a shared Docker network to allow containers to communicate:

docker network create symbolix

Shared Docker Volume

Create a shared Docker volume for persistent log storage:

docker volume create symbolix-log

Containerized Database

Build and run the MySQL database container:

docker build \
    -f .docker/database/Dockerfile \
    --no-cache \
    --rm -t symbolix-mysql:latest \
    .docker/database

docker run --name symbolix-mysql \
    -p 55590:3306 \
    --network=symbolix \
    --env-file .docker/database/.env \
    --restart=always \
    --detach \
    symbolix-mysql:latest

Initialize or reset the database as needed:

docker exec -it symbolix-mysql mysql -h localhost -u root -p -e "DROP DATABASE IF EXISTS symbolix;"
docker exec -it symbolix-mysql mysql -h localhost -u root -p -e "CREATE DATABASE symbolix;"

Containerized Redis

Run a Redis container for caching and message brokering:

docker run --name symbolix-redis \
    -p 55582:6379 \
    --network=symbolix \
    --restart=always \
    --detach \
    redis:7.4.1-alpine

You can flush all Redis data or check its status:

docker exec -it symbolix-redis redis-cli -n 0 FLUSHDB

Containerized SMTP

Run a local SMTP server for email testing:

docker run --name symbolix-smtp \
    -p 55583:1025 \
    -p 55584:1080 \
    --network=symbolix \
    --restart=always \
    --detach \
    dockage/mailcatcher:latest

Access the MailCatcher web UI at http://43.218.247.170:55584.

Containerized SFTP

Run a local SFTP server for file transfer:

docker run --name symbolix-sftp \
    -p 55585:22 \
    --network=symbolix \
    --restart=always \
    -d atmoz/sftp \
    'admin:Secret123!:::symbolix'

This command creates an SFTP container with full access for the admin user (password: Secret123!).

Install/Upgrade Development Kit

To install all dependencies and the kit in editable mode, run:

pip install -e .[all] --config-settings editable_mode=strict

This allows you to make changes to the codebase and have them reflected immediately without reinstalling.

Git Management

Precommit

To ensure code quality and consistency, run the pre-commit hooks before pushing your changes. This will automatically check and format your code according to the project's standards:

be-precommit

If any issues are found, fix them and re-run the command until all checks pass.

Commit Message

Follow the Conventional Commits specification for your commit messages. This standard helps automate versioning and changelog generation.

A conventional commit message consists of a type, an optional scope, and a concise description:

<type>[optional scope]: <description>

Types include:

• feat: A new feature
• fix: A bug fix
• docs: Documentation changes
• style: Code style changes (formatting, missing semi colons, etc.)
• refactor: Code changes that neither fix a bug nor add a feature
• perf: Performance improvements
• test: Adding or updating tests
• chore: Changes to the build process or auxiliary tools

Examples:

feat(auth): add JWT authentication
fix(database): resolve connection leak
docs(readme): update setup instructions

Refer to the Conventional Commits documentation for more details and examples.

Tagging a Release

Release and dev package creation is managed by CI/CD:

• Development Package: Pushing to the develop branch triggers CI to build and publish a dev package. Manually bump the version using bump2version [major|minor|patch|build] before pushing.

• Release Package: Creating or pushing a tag in GitLab will trigger CI to build and publish a release package. Normally, tags are created manually using:

  bump2version --tag release
  git push && git push --tags
  

  Make sure to push both commits and tags to ensure the release process is triggered.

Typical workflow:

1. Work on a feature or fix in a separate branch.
2. Merge your branch into develop, bump the version manually, and push.
3. When ready for release, merge develop into main. CI will handle the final version bump and package publishing.

Manually Publish a Package Release

To manually publish a new package release, ensure the following environment variables are set with the appropriate values in your .env file:

BE_CI_PACKAGE_REGISTRY=http://example.com
BE_CI_AUTH_USERNAME=string
BE_CI_AUTH_TOKEN=secret

These variables configure the package registry URL and authentication credentials required for publishing.

Next, build and push the package to the registry using:

be-publish

Debug Commands

• Check the number of database connections:

  docker exec -it symbolix-mysql mysql -h localhost -u admin -p -e "SHOW STATUS WHERE variable_name = 'Threads_connected';"
  

• Single endpoint load test using ApacheBench:

  ab -n 100 -c 10 -H "Authorization: Bearer [jwt]" http://localhost:[port]/api[service_shortname]/[endpoint]
  

These commands help monitor and test your services during development.