Search Results for "airflow"
Sort By:
Showing 23 open source projects for "airflow"
View related business solutions-
Gemini 3 and 200+ AI Models on One PlatformBuild generative AI apps with Vertex AI. Switch between models without switching platforms.
-
Catch Bugs Before Your Customers DoMove from alert to fix before users notice. AppSignal monitors errors, performance bottlenecks, host health, and uptime—all from one dashboard. Instant notifications on deployments, anomaly triggers for memory spikes or error surges, and seamless log management. Works out of the box with Rails, Django, Express, Phoenix, Next.js, and dozens more. Starts at $23/month with no hidden fees.
-
1
Apache Airflow
Programmatically author, schedule, and monitor workflows
Apache Airflow is a community-created platform for programmatically authoring, scheduling, and monitoring workflows. It’s much easier to do all these things when workloads are defined as code. They become more versionable, testable, maintainable and collaborative. With Airflow you can author workflows as directed acyclic graphs (DAGs) of tasks. -
2
Apache Airflow Provider
Great Expectations Airflow operator
Due to apply_default decorator removal, this version of the provider requires Airflow 2.1.0+. If your Airflow version is 2.1.0, and you want to install this provider version, first upgrade Airflow to at least version 2.1.0. Otherwise, your Airflow package version will be upgraded automatically, and you will have to manually run airflow upgrade db to complete the migration. This operator currently works with the Great Expectations V3 Batch Request API only. ... -
3
gusty
Making DAG construction easier
...All you have to do is provide a list of dependencies or external_dependencies inside of a task file, and gusty will automatically set each task's dependencies and create external task sensors for any external dependencies listed. gusty works with both Airflow 1.x and Airflow 2.x, and has even more features, all of which aim to make the creation, management, and iteration of DAGs more fluid, so that you can intuitively design your DAG and build your tasks. -
4
papermill
Parameterize, execute, and analyze notebooks
...Papermill supports both Python API usage and a command-line interface, making it flexible for integration with CI/CD systems, shells, and workflow orchestration tools like Airflow. -
Fully Managed MySQL, PostgreSQL, and SQL ServerCloud SQL handles your database ops end to end, so you can focus on your app.
-
5
Mage.ai
Build, run, and manage data pipelines for integrating data
Open-source data pipeline tool for transforming and integrating data. The modern replacement for Airflow. Effortlessly integrate and synchronize data from 3rd party sources. Build real-time and batch pipelines to transform data using Python, SQL, and R. Run, monitor, and orchestrate thousands of pipelines without losing sleep. Have you met anyone who said they loved developing in Airflow? That’s why we designed an easy developer experience that you’ll enjoy. -
6
Ploomber
The fastest way to build data pipelines
...Ploomber automatically manages task dependencies and execution order, allowing complex pipelines with multiple stages to run reliably. The framework can deploy pipelines across different computing environments including Kubernetes, Airflow, AWS Batch, and high-performance computing clusters. It also helps teams maintain reproducibility by tracking changes in code and rerunning only outdated pipeline tasks. -
7
Arkflow
High performance Rust stream processing engine
Arkflow is a Rust-based framework for building reactive, event-driven data pipelines. Inspired by tools like Airflow and Dagster, it focuses on strong typing, modularity, and performance. Arkflow is ideal for developers who want a fast, extensible way to orchestrate workflows and data transformations in Rust. -
8
SageMaker Python SDK
Training and deploying machine learning models on Amazon SageMaker
SageMaker Python SDK is an open source library for training and deploying machine learning models on Amazon SageMaker. With the SDK, you can train and deploy models using popular deep learning frameworks Apache MXNet and TensorFlow. You can also train and deploy models with Amazon algorithms, which are scalable implementations of core machine learning algorithms that are optimized for SageMaker and GPU training. If you have your own algorithms built into SageMaker-compatible Docker... -
9
prek
Better `pre-commit`, re-engineered in Rust
...It’s optimized for modern workflows, supporting monorepos with workspace modes, parallel hook installation and execution, and native handling of common hook tasks without external runtimes. Because it ships as a single binary with no runtime dependencies and is already being adopted in large projects (like CPython, Apache Airflow, and FastAPI), prek is increasingly seen as a next-generation tool for developers who value performance, simplicity, and robust git hook automation. -
Try Google Cloud Risk-Free With $300 in CreditUse your credit across every product. Compute, storage, AI, analytics. When it runs out, 20+ products stay free. You only pay when you choose to.
-
10
TFX
TFX is an end-to-end platform for deploying production ML pipelines
TensorFlow Extended (TFX) is a Google-production-scale machine learning platform based on TensorFlow. It provides a configuration framework to express ML pipelines consisting of TFX components. TFX pipelines can be orchestrated using Apache Airflow and Kubeflow Pipelines. Both the components themselves and the integrations with orchestration systems can be extended. TFX components interact with an ML Metadata backend that keeps a record of component runs, input and output artifacts, and runtime configuration. This metadata backend enables advanced functionality like experiment tracking or warm starting/resuming ML models from previous runs. -
11
Pyper
Concurrent Python made simple
Pyper is a Python-native orchestration and scheduling framework designed for modern data workflows, machine learning pipelines, and any task that benefits from a lightweight DAG-based execution engine. Unlike heavier platforms like Airflow, Pyper aims to remain lean, modular, and developer-friendly, embracing Pythonic conventions and minimizing boilerplate. It focuses on local development ergonomics and seamless transition to production environments, making it ideal for small teams and individuals needing a programmable and flexible orchestration solution without the overhead of enterprise systems. -
12
Airbyte
Data integration platform for ELT pipelines from APIs, databases
We believe that only an open-source solution to data movement can cover the long tail of data sources while empowering data engineers to customize existing connectors. Our ultimate vision is to help you move data from any source to any destination. Airbyte already provides the largest catalog of 300+ connectors for APIs, databases, data warehouses, and data lakes. Moving critical data with Airbyte is as easy and reliable as flipping on a switch. Our teams process more than 300 billion rows... -
13
Hamilton DAGWorks
Helps scientists define testable, modular, self-documenting dataflow
Hamilton is a lightweight Python library for directed acyclic graphs (DAGs) of data transformations. Your DAG is portable; it runs anywhere Python runs, whether it's a script, notebook, Airflow pipeline, FastAPI server, etc. Your DAG is expressive; Hamilton has extensive features to define and modify the execution of a DAG (e.g., data validation, experiment tracking, remote execution). To create a DAG, write regular Python functions that specify their dependencies with their parameters. As shown below, it results in readable code that can always be visualized. ... -
14
Peltier rack AC
Peltier based AC for 19" server rack 18U
The system is designed to cool a small 19" server rack where traditional cooling methods are impractical. In my case, it’s for a home lab setup in a 3m² storage room with forced exhaust only. A cooling block beneath the rack channels inlet airflow before it passes through the rack. Two 120mm PWM fans mounted on the top drive airflow, while a heat diffuser, consisting of a 240mm radiator with two 120mm PWM fans, is positioned outside the room to disperse heat. The system uses water as the coolant. System Components: - Cooling Block: Includes 8 Peltier elements, aluminum heatsink, and water blocks (1 block for 2 elements)... -
15
ransfoil
console to calculate airflow around an airfoil based on RANS approach
RANSFOIL is a console program to calculate airflow field around an isolated airfoil in low-speed, subsonic, transonic or supersonic regime by numerically solving the Reynolds averaged Navier-Stokes (RANS) equations using mature computational fluid dynamics (CFD) method. As postprocess results, the aerodynamic parameters of the airfoil, e.g., lift coefficient and drag coefficient, can be integrated from the airflow distribution near wall boundary. -
16
Tributary
Streaming reactive and dataflow graphs in Python
Tributary is a library for constructing dataflow graphs in Python. Unlike many other DAG libraries in Python (airflow, luigi, prefect, dagster, dask, kedro, etc), tributary is not designed with data/etl pipelines or scheduling in mind. Instead, tributary is more similar to libraries like mdf, loman, pyungo, streamz, or pyfunctional, in that it is designed to be used as the implementation for a data model. One such example is the greeks library, which leverages tributary to build data models for options pricing. -
17
aws-mwaa-local-runner
CLI that replicates an Amazon Managed Workflows for Apache Airflow
This repository provides a command-line interface (CLI) utility that replicates an Amazon Managed Workflows for Apache Airflow (MWAA) environment locally. The CLI builds a Docker container image locally that’s similar to an MWAA production image. This allows you to run a local Apache Airflow environment to develop and test DAGs, custom plugins, and dependencies before deploying to MWAA. To stop the local environment, Ctrl+C on the terminal and wait till the local runner and the postgres containers are stopped. ... -
18
Couler
Unified Interface for Constructing and Managing Workflows
...Couler endeavors to provide a unified interface for constructing and optimizing workflows across various workflow engines, such as Argo Workflows, Tekton Pipelines, and Apache Airflow. Couler enhances workflow efficiency through features like Autonomous Workflow Construction, Automatic Artifact Caching Mechanisms, Big Workflow Auto Parallelism Optimization, and Automatic Hyperparameters Tuning. -
19
cronsun
A Distributed, Fault-Tolerant Cron-Style Job System
...It's similar with crontab on stand-alone *nix. The goal of this project is to make it much easier to manage jobs on lots of machines and provides high availability. cronsun is different from Azkaban, Chronos, Airflow. cronsun has been tested in production for years on hundreds of servers. Although the current version is not released as an stable version, but we think it is completely available for the production environment. We encourage you to try it, it's easy to use, see how it works for you. We believe you will like this tool. cronsun support security with security.json config. ... -
20
Step-by-step instructions for making your HVAC (Heating, Ventilation & Air Conditioning) system state-of-art. Covers airflow balancing, temperature control & zoning, energy conservation measures, remote access, 1-wire, XBee devices, home automation
-
21
COMIS (Conjunction of Multizone Infiltration Specialists) is a multizone airflow simulation program.
-
22
Web based visualizations of data from CPAP/APAP machines, such as the Philips Respironics System One
-
23
Language: C# Framework:Microsoft .NET Framework Supported OS:Microsoft Windows 2000 and XP Features: Logging and display of Wideband O2, pressure, airflow, RPM, and throttle from the Emanage hardware. Much credit and thanks goes to Hau
- Previous
- You're on page 1
- Next
