Ox Pilot FAQ

Any system with an available API. Common limitations in vendor APIs can include things like:

• Some APIs are read only

• Some APIs are limited in terms of what data can be accessed

• Some APIs do not offer all the fields

Sort of. It is a set of tools which approximates a middleware tool. Our service is a combination of hosted Apache-Airflow, Github code repository and a Snowflake “data mart” (see Terminology section) to deliver task automation and data pipelines.

We always provision a new Snowflake instance for our Task Automation service to ensure proper serviceability by our team.

Yes. Apache Airflow is a very large open source community, an open source project and is very actively supported and updated. Find more details here and here.

We have worked in open source systems, both using as well as contributing, for most of our collective careers. We see open source products, when they are actively supported by the community, are longer lasting systems which reduce technology turn over and often are lower costs. 

Our current servers are at Release version 3.0.6. Here is a link to the Github repo for the Airflow project.

Yes. We host our Airflow servers at a top tier virtual server service located in Northern California.

Yes. We have 2 methods for backup:

• Our code is all controlled through a Github repository and CI/CD automation

• Servers are backed up daily with a rigorous retention protocol.

Our servers house no client data, so the only things needing backup are code and log files.

Sort of. Our methods, change management system and Team are the only proprietary aspects to our service. the core Apache-Airflow server is an open-source project.

No. It does not include it but it does require it. We walk the client through setup of a Snowflake account then we install our Data Mark footprint (see Terminology section). Costs for the Snowflake service typically range from $0 to $200 per month, depending on the amount of Compute you use from Snowflake. We design our automations to avoid using Snowflake for anything other than storage.

Yes. We provide 95% uptime on the services and 2 hour response (during business hours) and resolution on any system failures we create. We have a rapid response SLA as well, let us know if you want more details on that premium service level.

Our service does not work with systems that do not have an API.

• First step is to provision a unique Integration user on the various systems

• Second step is to establish a connection from Airflow to the system

• Third is to setup monitoring

The Task Automation system is monitored by a combination of systems and our team. The primary tools used for Monitoring are Uptime Kuma (yup, another open source tool) in concert with our internal Slack system. We configure an Ox Bot channel for our clients so that they can see any alarms or updates in real time.

Messaging is typically via Slack or Teams using our “Ox Bot” channel.

Direct (read only) access to the Airflow Server and your workloads (“DAGs”, see terminology section of FAQ) We will be providing a read only access method to the Airflow interface which will allow clients to see their DAGs, task runs

<thumbnails here>

We provide two types of ongoing services:

• Airflow task automation
• Datamart ongoing support (e.g. curation, cleaning, mining, standardizing, etc)

Yes. Onboarding of a new client involves 10 or more hours of listening to the business needs, assessing the technical requirements, and also guiding clients on how to setup a “Data Team” to enable their business to collaborate with our service team.

We work with our clients’ internal Change Management methods and procedures and aim to make the relationship collaborative but also flexible.

We charge a fixed annual flat fee for the core automation service (billed at sign-up) which includes an initial 10 hour design/build engagement and gets you started with some automations.

Snowflake costs are the only variable costs in our service model and often are $0 because we design automations to use our Compute (see terminology section) and only use Snowflake for storage. Our clients have a direct vendor relationship with Snowflake and allow us access for our Airflow service.

We sell consulting hours in time blocks of 5 hours for $600 for change management and new automation builds.

We give it all to you. We maintain all DAG code in a Github repository. When you cancel the service we add you to your specific Github repos as read only then you are free to fork the code and own it.

We have worked in the data systems space for many years and worked with most middleware and integration tools (SSIS, Mulesoft, Dataloader, Talend, Workato, Boomi, Informatica, dbAMP, Zapier, etc).

We realized our clients automation that work well, and access to a fractionalized technical team for both ongoing maintenance but also change management as their environments change over time.

By working with Airflow we can pass along development cost savings to our clients and focus on maintaining those automations, but also building our understanding of your business and opportunities for new efficiencies.

Our services are a “White Glove” combination of Airflow as a Service, monitoring, and account management which folds in both design assist as well as data systems management guidance when needed.

Vendors control their APIs. Most vendors publish a “Version Life Cycle” (here is Ariflow’s) which gives you visibility into which release versions will be live and maintained, then sunsetted when, etc. The various systems and their APIs all are in a constant state of change. Our operations team monitors all these changes and as we move ahead in any one year, we plan and adapt as needed.

Some API change can be a surprise, and in those cases our developers can react quickly.

TBD

TBD

Our pilot clients will have 2 methods for getting support:

• Slack or Teams “OxBot” – a channel where you can ask questions and get answers. 
• Email support with Zoom meetings

Your support team would handle status feature enhancements and change management

White glove team will be your interface

OxBot

• When a client purchases our service, we walk through these typical stages:
• What do you see in the initial steps
• What can you expect
• What if we don’t

We do the following for you, so you and your technical team do not have to:

• Run and maintain multiple Airflow servers (SQL, Python, etc)
• Manage code and CI/CD for change management
• Work with your systems’ APIs and build and maintain automations

We have been in business for 20+ years and have seen many data system consolidations, conversions, integrations and automations, and Airflow has become a favorite of ours.

As a business and a technical team, we are all dedicated to providing very high quality service at a great price. We do this by being very focused on one thing – Task Automation with Airflow.

TBD

TBD

TBD