With the release of Kubernetes 1.8, Cronjobs have graduated to beta, which means we can now more easily run Neo4j backup jobs against Kubernetes clusters.
Before we learn how to write a Cronjob, let’s first create a local Kubernetes cluster and deploy Neo4j.
This post is part two of a two-part series. Containerizing Databases at New Relic, Part 1: What We Learned looked at four inherent challenges of putting databases in Docker containers and how New Relic addressed them.
The tooling required to manage a large number of small databases is different than the tools needed to manage a few large databases. At New Relic, we've constructed a management framework for small- and medium-sized databases that we call Megabase. Megabase consists of microservices and tools written in Go that enable us to deploy replicated groups of containerized database instances to multi-tenant hosts. Megabase currently supports recent versions of MySQL (via Percona) and PostgreSQL. We're in the process of adding support for Redis.
Microservices can decrease the risk of experimenting with new technologies and can enable each part of an application to scale at its own pace. Of course, each microservice should be as independent as possible from the other microservices in an ecosystem, ideally only connected by a communications interface. Powering your microservices with technologies that are both versatile and resource lean can help achieve this independence.
Open-source Tarantool is one of the most microservice-ready databases on the market due to its small footprint and unique architecture. It can execute an enormous amount of work on a single core, which means that it can run effectively on everything from an IIoT device to a small cloud VPS instance to a multicore server. The latter could actually run a whole suite of Tarantool services. And the fact that Tarantool has all of the features that it needs to function as an independent database means that any microservice it powers doesn’t require an additional database (as it would, were Tarantool a more conventional cache).
Apache Cassandra has become an incredibly popular database for software architects and engineers. Many of us trust it for our applications’ data and, presently, there are thousands of deployments running this reputable NoSQL database. No doubt, Cassandra totally deserves its glory and reputation. The database simply has what it is expected of it: unlimited scalability and high-availability with fast write operations.
These two essential capabilities helped Cassandra rapidly rise in popularity by solving problems and use cases that relational databases could not tackle. The problems and use cases required horizontal scalability, high-availability, fault-tolerance, and 24/7 operability without downtime. Classic relational databases could not fulfill all of the requirements then — and cannot now (except perhaps for distributed relational databases such as Google Spanner or CockroachDB).
Thanks to Robert Reeves, Co-founder and CTO at Datical, for taking me through their collaboration with Pivotal Software, Inc. on a container image for Concourse, enabling enterprises to speed their full stack cloud-native application delivery process by automating database changes alongside application changes.
Concourse is a Continuous Integration/Continuous Delivery (CI/CD) open-source project sponsored by Pivotal geared for teams that practice Agile development. Concourse is a popular choice among Pivotal Cloud Foundry® (PCF) users for Continuous Delivery because of its container-based builds, flexible integrations, and visual dashboards.
I travel a lot on business as a Developer Advocate for Redis Labs! I'm not much of a car guy, so when I have some free time, I prefer to walk or bike around a city. Many of the cities I've visited on business have bike share systems that let you borrow a bike for a few hours. Most of these systems have an app for renting bikes, but they only share the details of their system. This got me thinking: using publicly available bike share information to build an "app" showing you global information would be a fun way to demonstrate the geospatial features of Redis. With that GeoBike, the Redis bike share application was born.
GeoBike incorporates data from many different sharing systems, including the CitiBike Bikeshare in New York City. We are going to take advantage of the General Bikeshare Feed provided by Citi Bike system and use their data to demonstrate some of the features we can build using Redis to index geospatial data. The CitiBike data is provided under the NYCBS Data Use Policy.
There's a saying: "In Silicon Valley, you can throw a rock in any direction and you'll probably hit a software engineer." The other day, I was traveling by train from San Jose to San Francisco when I overheard two software engineers comparing the speed of the products they were developing. When I heard one developer say, "Our app runs fast because we cache all the session data," I realized that even a Silicon Valley engineer may be unaware of the subtle differences between a cache and a session store. Redis, as an in-memory database, is used for both caching and session store scenarios. Let's look at how the use cases differ.
CacheAn application stores data in the cache to serve future requests faster. Typically, the cache storage is located in the RAM and has sub-millisecond latency.
This post will walk you through each step of creating interactive, real-time, and dynamic dashboards to monitor your MySQL instances using Telegraf, InfluxDB, and Grafana.
Start by enabling the MySQL input plugin in /etc/telegraf/telegraf.conf:
Welcome back! In my last Database deep dive, we talked about database administration and all the things you need to know about it, such as what the future holds for DBAs and how to work Agile methodologies into your database administration practices. This time around, since we're wrapping up 2017, we'll be looking at some of the most popular database articles on DZone from this year, the top news that happened in the world of databases, and more. We'll also, as always, provide you with some more detailed resources and job information so that you can start your new year full of knowledge and start effectively hunting for that job that really makes you happy.
Happy Database Days on DZoneCheck out some of the top Database Zone articles on DZone that were published in 2017. Here, you'll find everything from stories of switching databases to the pros and cons of normalization.
Last time, we learned how to get Flask set up. In this article, we will learn how to add a database to our music data website. As you might recall, Flask is a micro-web-framework. That means it doesn't come with an Object Relational Mapper (ORM) like Django does. If you want to add database interactivity, then you need to add it yourself or install an extension. I personally like SQLAlchemy, so I thought it was nice that there is a ready-made extension for adding SQLAlchemy to Flask called Flask-SQLAlchemy.
To install Flask-SQLAlchemy, you just need to use pip. Make sure that you are in your activated virtual environment that we created in the first part of this series before you run the following or you'll end up installing the extension to your base Python instead of your virtual environment:
As cloud and container-based environments become ubiquitous, organizations are increasingly moving away from high-performance static hardware to dynamically changing, virtual, commodity infrastructure. Systems need to expand and contract easily to meet an ever-changing demand — and that includes the database.
Database scaling is often implemented by clustering. Here's a quick overview of the pros and cons of traditional architectural implementations:
There are multiple causes of the ORA-01745 ("invalid host/bind variable name error") error when using an Oracle database. The Oracle 9i documentation on errors ORA-01500 through ORA-02098 provides more details regarding ORA-01745. It states that the "cause" is "a colon in a bind variable or INTO specification was followed by an inappropriate name, perhaps a reserved word."
It also states that the "action" is "change the variable name and retry the operation." In the same Oracle 12g documentation, however, there is no description of "cause" or "action" for ORA-01745, presumably because there are multiple causes and multiple corresponding actions associated with this message.
I was told that putting more than a single exclamation mark on the title is in bad taste, but it was really hard to refrain.
Yesterday was my birthday and I celebrated with no remaining issues for RavenDB 4.0.
Spatial queries are fun when you look at them from the outside — but not so fun when you are working to implement them. But that is probably not your concern.
RavenDB has had support for spatial queries for many years now, but the RavenDB 4.0 release has touched on that, as well. And now, you can query spatial data with much greater reach. Here is a small sample of how this works:
NoSQL databases have become more popular in several areas. NoSQL has many different uses, including graph use cases. The graph database has a structure pretty different from relational technology and has a lot of successful cases, such as recommendation systems on Facebook and LinkedIn. This article will cover what a graph database is and how to use it with Java.
The Graph Structure
A counter table is a table that keeps counts of particular items or of certain keys. This can range from page count on your blog to keep track of the limit the user is allowed to have in terms of a particular item or service.
Usually, a counter table would be better kept in something like Memcached or Redis, as frequent increment updates are better-suited to those in-memory systems.
Sometimes, there is no escaping it and you must accept some user’s input into your system. That sad state of affairs is problematic because all users have one common tendency: they are going to be entering weird stuff into your system. And when stuff doesn’t work, you’ll need to make it work, even it makes no sense.
Let's take a simple example with this user, shall we?
Previously, I used the LTREE extension to save a tree data structure in a Postgres table. After saving the tree, I used the @>, or ancestor operator, to count the number of descendant nodes on a given branch.
But that's not all LTREE can do. Today, I'll show you how to delete, move, and copy branches from one place to another in your tree using @> in combination with other LTREE functions. After that, in my last post in this series, I'll look at how LTREE works under the hood and explore the computer science that makes all of this possible.
We are super excited to announce the general availability of Redis Enterprise 5.0. This latest software pack is now available with new modules on our downloads page.
With the 5.0 release, Redis Enterprise provides higher availability, scaling, performance, and unmatched cost efficiency over open source Redis.
You are probably sick of me saying it, but one of the things I love about Neo4j is that you can customize it any way you want. Extensions, stored procedures, plugins, custom indexes, custom apis, etc. If you want to do it, then you can do it with Neo4j.
So, the other day, I was like, what about this gRPC thing? Many companies standardize their backend using RESTful APIs, others are trying out GraphQL, and some are using gRPC. Neo4j doesn’t support gRPC out-of-the-box, partially because we have our own custom binary protocol “Bolt,” but we can add a rudimentary version of gRPC support quite easily.
You’re in your new job as a database administrator or data engineer and you just got lost trying to figure out what these insane looking queries are supposed to mean and do. Why are there five joins and why is there an ORDER BY used within a subquery before one of the joins even happens? Remember, you were hired for a reason — most likely, that reason also has to do with many convoluted queries that were created and edited over the last decade.
ExplainThe EXPLAIN keyword is used throughout various SQL databases and provides information about how your SQL database executes a query. In MySQL, EXPLAIN can be used in front of a query beginning with SELECT, INSERT, DELETE, REPLACE, and UPDATE. For a simple query, it would look like the following:
The in-memory powerhouse Tarantool supports some of the largest and most sophisticated enterprises in the world. But every developer using it had to sit down one day and answer the simple question, “How do I actually get this thing running?” This guide covers that step, i.e. the absolute first one: where to download Tarantool, how to start it up, and a little bit of what you can do with it.
I have based the guide on a Digital Ocean Ubuntu droplet for consistency across platforms, but you could use another cloud service or even a local installation with some tweaks. Also, cutting and pasting doesn’t necessarily encourage learning, but I wanted to point out that if you are in a rush, you could just cut and paste your way to a basic Tarantool HTTP server instance in less than 10 minutes.
The “identity” of the binlog server layer from the slave server’s point of view is something that could be modified in order to provide all slaves such common parameters for all MariaDB MaxScale servers they could replicate from.
This way, slave servers can see the same values for such config options and they will not be able to understand whether the real master has changed after a failure or a new server has been promoted as master, for any reason.
Suppose you had a hierarchical data structure in your application. How would you save it in a database? How would you represent a complex tree structure using flat rows and columns?
There are a few different, equally valid, options. In this series of blog posts, I'll take a close look at one option that Postgres provides: the LTREE extension. If you install and enable LTREE on your Postgres server, it will add powerful SQL operators and functions that support tree operations.
Thanks to David Thompson, V.P. of Engineering at MariaDB® Corporation, for taking me through the thinking of their recent integration with Tableau Software.
The certified MariaDB integration with Tableau’s business intelligence (BI) and visual analytics platform brings together data management products and visualization technologies to enable businesses to use these solutions for reliable, fast, data-driven business decisions.
This post is a hands-on look at ZFS with MySQL.
In my previous post, I highlighted the similarities between MySQL and ZFS. Before going any further, I'd like you to be able to play and experiment with ZFS. This post shows you how to configure ZFS with MySQL in a minimalistic way on either Ubuntu 16.04 or Centos 7.
ACCS provides a pre-configured platform (Platform-as-a-Service, or PAAS) where you can quickly deploy and host your applications. For many of today's applications, the hosting server is just that: a place to host the application. Most of the time, the only thing an application needs from the server is to have it support the application's programming language and to provide in-and-out connections through ports. Using a PAAS such as ACCS frees you from all of the extra work of configuring and maintaining a server and allows you to focus on perfecting your application.
ACCS supports multiple languages but for this post, I'll focus on Node.js.
With all the hullabaloo with NoSQL companies going public and then losing a third of their valuation, being bought out, and some even going into administration and then being resurrected by customers — you could be forgiven for thinking the NoSQL bubble has spectacularly burst.
It’s crunch time for NoSQL companies, that’s for sure. They need to grow up. Rapidly. They need to become easier to deploy, use, and manage. But they’re working on that.
Being able to execute full-text search queries in Couchbase without the need for additional tooling such as Elastic is huge for NoSQL.
About a year ago, I had written about using full-text search (FTS) in Couchbase Server with the Node.js SDK. This is back when FTS was in developer preview. While still very valid, it doesn't encapsulate the true power of what you can do with full-text search. Take facets, for example. Facets are aggregate information collected on a result set and are useful when it comes to categorization of result data.
The in-memory powerhouse Tarantool supports some of the largest and most sophisticated enterprises in the world. But every developer using it had to sit down one day and answer the simple question, “How do I actually get this thing running?” This guide covers that step, i.e. the absolute first one: where to download Tarantool, how to start it up, and a little bit of what you can do with it.
I have based the guide on a Digital Ocean Ubuntu droplet for consistency across platforms, but you could use another cloud service or even a local installation with some tweaks. Also, cutting and pasting doesn’t necessarily encourage learning, but I wanted to point out that if you are in a rush, you could just cut and paste your way to a basic Tarantool HTTP server instance in less than 10 minutes.
ASP.NET Core is the newest development platform for Microsoft developers. If you are looking for information about plain old ASP.NET, check out ASP.NET With Couchbase: Getting Started.
ASP.NET Core Tools to Get StartedThe following video will take you from having no code to having an HTTP REST API that uses Couchbase Server, built with ASP.NET Core.
Recently, I was investigating an interesting case for a customer. We could see the regular spikes on a graph depicting "InnoDB rows inserted" metric (jumping from 1K/sec to 6K/sec); however, we were not able to correlate those spikes with other activity. The
Other graphs (Com_*, Handler_*) did not show any spikes like that. I've examined the logs (we were not able to enable general log or change the threshold of the slow log), performance_schema, triggers, stored procedures, prepared statements and even reviewed the binary logs. However, I was not able to find any single write query that could have caused the spike to 6K rows inserted.
The Oracle Database supports a concept of a null value, which means, essentially, that it has no value. The idea of having nulls in a relational database is controversial, but Oracle Database supports them and you need to know how they can impact your work in PL/SQL.
First, and most important, remember that:
In my role as a Sales Engineer here at InfluxData, I get to talk to a lot of clients about how they're using InfluxDB and the rest of the TICK Stack. We have a large number of very big clients using InfluxEnterprise for metrics collection, analysis, visualization, and alerting in their DevOps area, so we've done a lot of scale-out testing for these clients. In these tests, we see very linear scale-out as we add additional nodes to an InfluxEnterprise Cluster. I'll talk about this in my next blog post.
Over the last six months, I've seen more and more large manufacturers, energy companies, and utilities coming to us for collecting metrics from their IoT devices. Many times, they're working with consulting companies that specialize in building IoT solutions, and these companies bring InfluxDB into the solution because we're so well-suited for time series applications.
The columnar storage technique has proven effective in handling many computing scenarios and is commonly adopted by data warehousing products. The technique is usually a synonym of high-performance in the industry.
Yet the strategy has its own weaknesses. A Google search shows that criticisms surrounding it are mainly about data modification. There are few discussions of its application to read-only data analysis and computing, which will be taken care of in the following.
Migrating a MySQL database usually requires only a few simple steps, but can take quite some time, depending on the amount of data you would like to migrate.
Tarantool was created by hardcore developers who maintain web services that process hundreds of thousands of requests per second. But how can such a powerful tool help regular programmers?
Let’s create a simple but amusing service capable of handling heavy workloads.
A graph database is a data management system software. The building blocks are vertices and edges. To put it in a more familiar context, a relational database is also a data management software in which the building blocks are tables. Both require loading data into the software and using a query language or APIs to access the data.
Relational databases boomed in the 1980s. Many commercial companies (i.e. Oracle, Ingres, IBM) backed the relational model (tabular organization) of data management. In that era, the main data management need was to generate reports.
A shorter post this time around — it feels nice to get something finished faster than two weeks for once. Today, we will look at startup and shutdown scripts in Spring Data Cassandra. This is something I probably should have done myself ages ago, as it would have made testing my earlier posts much easier. I spent so much time (slightly over exaggerated) constantly truncating tables between each execution, which was pretty annoying.
The content in this post is related to my earlier posts on Spring Data Cassandra but does not directly require them to be read. That being said, it is worth looking at my post Getting Started With Spring Data Cassandra for a basic understanding of the subject. The dependencies can also be found there.
Amazon Web Services (AWS) is a cloud platform that offers a wide variety of services, including computing power, database storage, content delivery, and other functionality that targets businesses of all sizes. One of their database solutions includes the Amazon Relational Database Service. Amazon RDS includes a number of popular RDBMSes, including Amazon Aurora, PostgreSQL, MySQL, MariaDB, Oracle, and Microsoft SQL Server, as well as tools to manage your databases and monitor their performance.
Despite the wide range of metrics available within the Amazon RDS console, there are some very good reasons for using your own monitoring tool(s) instead of or in addition to those offered by Amazon RDS. For example, familiarity with your own tool(s) or access to features that Amazon RDS does not provide would constitute two persuasive reasons for employing a local tool.
We're once again running our popular SQL in the City event online so that the Redgate and data platform community can benefit from free training in a virtual setting.
On December 13, we'll be live-streaming from Redgate HQ from 11 AM - 4 PM GMT, and again from 6 PM - 11 PM GMT. Technical sessions will dive into the latest Microsoft SQL Server releases, and cover topical issues such as DevOps, data compliance, protection, and privacy.
Given how fast technology is changing, we thought it would be interesting to ask IT executives to share their thoughts on the biggest surprises in 2017 and their predictions for 2018.
Here's what they told us about databases:
DinoDate currently has both Python and NodeJS mid-tier applications and is backed by an Oracle Database.
The following instructions show how to deploy DinoDate to an Oracle Cloud Compute instance. However, if you just need to deploy a NodeJS or Python application, the same instructions should help you install Node and/or Python 3.
In SQL Server database, transaction logs play a vital role. Whenever a system failure or database corruption occurs in SQL Server, the transaction logs can be helpful for recovery. SQL Server stores all the transactional information of each database in LDF (.ldf) files.
In today’s post, I am going to discuss transaction logs and LDF files. Also, we will learn how to read data from LDF files in SQL Server.
To make sure you're keeping up, check out our previous articles on Neo4j and the Paradise Papers before reading this one!
The cryptocurrency market is hotter than ever. Bitcoin and Ethereum are two of the most popular cryptocurrencies out there, and their price is heavily influenced by speculation fueled by news articles and public statements from thought leaders. To help you keep track of the ever-changing world of crypto, I'm going to walk you through how to build a custom service input plugin for Telegraf that will monitor keywords from a Twitter live stream and send them to InfluxDB. We'll use Chronograf to explore the data and build a simple dashboard. Finally, we will end up with a dashboard that looks similar to the following.
Of course, all of this is just for fun unless you are one of the lucky few that has access to the Twitter firehose — but it's a good way to get introduced to writing a custom Telegraf plugin for whatever system or device you need to monitor.
I am (finally) diving in to explore and then write about using JSON in Oracle Database 12c with SQL (Releases 1 and 2) and — more to the point for me — with PL/SQL (based on enhancements in Release 2).
Expect to see posts on my blog, articles in Oracle Magazine, quizzes and workouts at the Oracle Dev Gym, and more in 2018.
Apache Cassandra is getting more and more popular these days. Some companies may want to operate a multi-tenant cluster. What options does Cassandra provide for this? Do you have your services prepared to do the work properly?
In this article, we will go through a practical example using Spring Boot and spring-data-cassandra.
DZone Database Zone Fun With SQL: Functions in Postgres In our previous Fun with SQL post on the Citus Data blog, we covered w...
