Developing for MyTardis
Open source and active developer community
The source code and all collaboration is hosted on GitHub.
MyTardis has been developed as an open source project by an active community of developers from research institutions in Australia and around the world.
Built on modern web technologies
MyTardis mostly written in the Python programming language and is built on top of the Django web framework. A complete installation of the service also includes an Elasticsearch index, a RabbitMQ-based task queue, an Nginx server, and a PostgreSQL database.
To set up and manage these services we employ the SaltStack orchestration software and cloud technologies.
Plugin architecture for data types and custom views
From the beginning, MyTardis was built to support custom file formats. On ingestion of files, custom filters can do work on them, for example to extract metadata or create preview images from discipline specific data, e.g. from microscopy images.
In addition to automatic previewing of images, custom front end code can be added for the preview or display of different file types.
Any storage backend
All data storage interaction is handled through the Django File storage API. This means that any storage backend with an appropriate adapter can be used to store files. Currently used or tested backends include:
- Local disk storage
- Mounted network drives (NFS, CIFS, SSHFS)
- S3 object storage
- Swift object storage
- SFTP accessible locations
Scalable service using cloud patterns
Thanks to the distributed architecture and modern technologies the service is easily scalable to the required workload.
Depending on the needs MyTardis can run on a single cloud instance or distributed across several data centres. It can be run in one or more Docker containers as well.