Shopify to Attend Percona Live 2018!
Share
2 min read
We’re excited to announce that we’ll be attending Percona Live 2018 in Santa Clara this coming April! Running from April 23-25, the Percona Live Open Source Database Conference is the premier event for anyone that develops and uses open source database software.
We’ll be sending three Shopify speakers to the event. Check out their conference topics below!
- Hiring DBAs from an HR perspective (Lightning Talk) - Kayla Boyer
The role of the DBA is evolving, as more companies and products move to a hybrid model where, along with their traditional work, DBAs are expected to code. We’ve scaled our Data Stores team to bring in pure developers, pure DBAs, and a mix of both. We were able to find creative ways to help support our DBAs to better adapt to the changes in the industry. We’re discovering how increasingly complex it is to find candidates with this mix of experience, and through learning from our struggles we have learned the best methods to find DBAs and to help them evolve. We want DBAs to be prepared for this new world and would love to share our industry findings with this community.
Monday, April 23 (4:30pm - 6:30pm)
- Automatic Failovers with Kubernetes using Orchestrator, ProxySQL and Zookeeper - Jordan Wheeler & Sami Ahlroos
In this session, we will discuss our fully automated failover solution running in containers on Kubernetes. Using Orchestrator for MySQL failovers, ProxySQL to route queries and a Zookeeper-backed application we wrote called Taiji for service discovery, database failures and topology changes are handled without any human intervention. This system is tolerant to network partitions and connectivity issues, node failures, and even full-region outages.
After adding additional functionality to Orchestrator, we have it deployed with the raft consensus protocol and automatic failovers enabled. ProxySQL is deployed alongside a Taiji container that watches for changes in Zookeeper. All topology changes are automatically pushed to Zookeeper via Orchestrator callback scripts and a Taiji agent that performs health checks on databases. In less than a second, these changes are pushed to ProxySQL, so our application will seamlessly begin sending read and write queries to the proper database.
Tuesday, April 24 (3:50pm - 4:40pm)
Existing tools like mysqldump and replication cannot migrate data between GTID-enabled MYSQL and non-GTID-enabled MySQL - a common configuration across multiple cloud providers that cannot be changed. These tools are also cumbersome to operate and error-prone, thus requiring a DBA’s attention for each data migration. We introduced a tool that allows for easy migration of data between MySQL databases with constant downtime on the order of seconds.
Inspired by gh-ost, our tool is named Ghostferry and allows application developers at Shopify to migrate data without assistance from DBAs. It has been used to rebalance sharded data across databases. We plan to open source Ghostferry at the conference so that anyone can migrate their own data with minimal hassle and downtime. Since Ghostferry is written as a library, you can use it to build specialized data movers that move arbitrary subsets of data from one database to another.
Tuesday, April 24 (4:50pm - 5:15pm)
Interested in working at Shopify? Talk to Kayla at the conference, or reach out on LinkedIn! You can also check out our careers page.