Being a developer in the current world means dividing products into interconnected microservices—standalone, always-up programs meant to address a few key functions in an assembly line of such services. The trouble with this pattern, however, is that much of your setup code is reinvented based on the last service you wrote. Then, by the time everything is up and running you find you burned as many cycles coaxing your program to work and play nicely with others as actually writing the key functions. Wookiee seeks to cut down on the time and clutter of building a communicative microservice architecture.
Wookiee is an open-sourced microservice framework that takes the error prone process of initial setup out of the picture. When I go to setup my next project, I take for granted that I will be ready for the main logic in a few lines of setup. I know that base Wookiee enables the hosting of health checks, recording/sending metrics to Graphite, and loading/updating global configurations. On top of that, it also has multiple Components that seamlessly incorporate the latest technologies. Java-based frameworks like Spring could fill a similar niche, but the flexible and extensible nature of Wookiee has allowed me to apply or temper it to every type of Service need encountered.
I will be walking you through the primary interfaces and key concepts of Wookiee. All code is in Scala but I hope it should be understandable with Java knowledge. In some places, I have simplified to avoid too many Scala or Akka specific concepts—but these should be apparent when actually setting up Wookiee. If you do implement Wookiee on your own, it will be critical to understand Akka Actors as Wookiee provides/manages the Actor System and its respective routing. If you see the word “Harness” in class or package names, it is because that was the original name before it became Wookiee (after the utility their species finds in harnesses).
The Wookiee Service
The Service is the root of your application, it is your access point to add your key functions and lets you branch out into using Wookiee’s features:
wookiee-system {
services {
internal = "MyMicroService"
}
}
Then build up your project and launch it by pointing to the Java “main()” containing HarnessService…
INFO MyMicroService – The service MyMicroService started
INFO ServiceManager – Service Manager started
INFO ServiceManager – Wookiee Started, Let’s Go
…and the Service is up. Now you are all setup and ready to start coding. You can even connect a companion test library that will let you spin up a mock Wookiee Service for build-time integration testing. Now you have logging access everywhere, HTTP accessible health checks and pings, configuration loading/watching, local Akka messaging, and helpful utility functions. Next, there is all the extra functionality we can add with Components.
Wookiee Components
Components are modular living libraries that spin up before your service and allow you to tap into a number of other libraries or specific features or technologies. A Service can have any number of Components, and a Component can be designed to do anything. We have already Open Sourced a few that utilize some of the latest technologies or add helpful features like metrics. In the below simplified Component, we create a connection to a database and receive queries with it:
class MyDBQueryComponent(name: String) extends Component(name) {
val dbConnection = DBConnection(config).start()
def receive = {
case DBQuery(queryString) =>
// run the query we were sent against the database connection we initialized
val dbQueryResult = dbConnection.query(queryString)
sender ! dbQueryResult
}
}
In the above example we establish a connection to our database, dbConnection, and are now ready to handle queries from the Akka Actor receive block (which inflows messages)—any results return to their original sender, which could exist anywhere in your Wookiee Service. There are many other possible applications of Components and they begin to become recognizable as you envision Wookiee-based architectures.
To imagine your next Component, think back to a time you went to build a service that was accessible externally via HTTP. First, you selected a library that fit your needs such as Spray. Next was setting up and configuring an HTTP server class which needed passing around, and most HTTP libraries would require you to compose complex routing trees that are difficult to split across classes. On top of everything, you have three more services to make before your product works end-to-end, some of which will need HTTP Servers of their own. This is the perfect place to consider putting all of your interactions with the chosen library into its own Component.
Beyond just a normal library, Components can be built to do things normally reserved for the service-specific startup logic. For instance, starting up technologies or connections to servers—or even existing as their own pseudo-service running alongside your service and taking messages to process and respond to independently.
The possibilities for Components are unlimited. Some uses we’ve realized; A connection to a SQL server that robustly queries a DB and responds, an interface that allows easy APIs and a managed connection to a server health monitoring tool like Nagios, or a Component that connects to Spark to read your own no-SQL Big Data stores allowing any Data Scientist to have democratized access. As the next evolution of libraries, Components seek to widen your Applications.
Wookiee Commands
When communicating in Wookiee, the first step is the Command. As with Services and Components, it is as easy as extending an interface to make your class into a Command. The below snippet is simplified (the actual version allows for more flexible marshalling and asynchronous processing):
class MyStringCommand extends Command {
def commandName: String = “StringCommand”
def execute(bean: CommandBean): CommandResponse = {
CommandResponse("Replying")
}
}
The MyStringCommand, like all other Commands, is setup to handle messages sent to it by keying off its commandName. It then processes them in the execute method and returns a response to whoever sent it. These messages can be from some external HTTP/Websocket entity, another Wookiee Service, or even internally from another class. Replying is optional and commands can process more than one request at a time—acting like a thread pool for parallel processing.
To communicate between services we depend on Zookeeper and Akka Remote, which are tools for registering cluster state and sending messages between server nodes. We wrap these tools using the Discoverable Command interface, which enables Commands to be seen by other Services and have messages exchanged. Think of it as being able to call a function in a class running on a different server. Getting this functionality is easy as using a different extension:
class MyStringCommand extends Command with Discoverable
The Command is now ready on its side to accept orders from other Services. The only thing that you have left to connect our commands is to make calls to them. The CommandHelper and its extension the DiscoverableCommandExecution (from the Wookiee Zookeeper Component) allow you to make calls to remote and local commands, respectively:
class MyStringCommandCaller extends Actor with DiscoverableCommandExecution {
…
executeCommand(“StringCommand”, bean)
executeRemoteCommand(“/other/service/path”, “StringCommand”, bean)
…
}
In addition, Commands themselves are accessible through HTTP or Websockets using one of our premade Components supporting HTTP (Colossus, Akka HTTP, Spray, and Socko currently supported). In these cases, it is usually just as easy as adding one more header onto your class and specifying the endpoint that the Command should match. In the below example you would be able to hit the execute method of the MyHttpAccessibleGetCommand by sending an HTTP request to your Service on the “/endpoint/to/match/over/http” path.
class MyHttpAccessibleGetCommand extends Command with AkkaHttpGet {
override def path: String = "endpoint/to/match/over/http"
…
}
Using a Service with Commands is a great way to connect your microservices and all of their functionality. Commands take advantage of the flexibility of Akka Actors to route requests and messages cluster-wide and are usually the external point of entry for any processing on your service.
A Wookiee about the Galaxy
As my organization has taken Wookiee into their architecture, it has had to adapt to running in different ecosystems and under adverse needs. The flexibility of Wookiee extends to its execution possibilities as well. For instance, Wookiee is right at home starting a Service as a managed “job” in a Hadoop YARN cluster. This opens up the possibility of Wookiee to interact with Spark via streaming or querying. Another example would be on a Docker Container. The Service model easily allows one to package their code into a Jar File for hosting on a Container with all its dependencies. To enable best practices, I have also built out a Wookiee Test repository allowing one locally to spin up a Mock Wookiee at build time for integration-like testing. This all in addition to the standard execution model of invoking Java that covers most use cases.
In speaking about Commands I addressed communication between Wookiee Services through Remote Commands. It is possible to go a step further in connecting our applications using the Cluster Component. This open-sourced Component allows you to send messages on topics that broadcast to all Services that have subscribed to that topic. The result could be dozens of interconnected Services constantly sending notifications and state changes between each other. That level of complexity—achieved with only a few dependencies and configurations.
Letting the Wookiee Win
Here at the Oracle Marketing Cloud we face diverse engineering challenges from many-user Application administration to very demanding Big Data collection and querying. Certain architectures do not lend themselves to Wookiee, such as data pipeline processing jobs running through a coordinator like Samza or Storm, as they provide much of the same initializing and supervising functionality. However, in every other case the business needs of the next project fit into the Wookiee Service model.
Imagine having a common vocabulary to discuss all of the entry points into your codebase. When everything is a Service, and a bug comes in with issues on the “/create/user” endpoint, then even an engineer who has never seen the codebase will know exactly what to search for (a Command who’s path matches the endpoint). The next time someone tries to find out what your program does, that developer will know to start at your Service class and analyze out with confidence. Whenever your team goes to add a new third party library, it will become common practice to create a Component to take care of all that technology’s setup and gotchas—so that future developers can skip straight to the benefits. During adoption, despite incessant pressure from strong business competitors and client demands, the Service model was so beneficial that we found the time to redesign our dozens of microservices. The framework itself has its quirks and still has room to grow, but its basic promise or allowing you to build anything with a universally applicable toolset stands firm.
Find Wookiee on Oracle’s GitHub (https://github.com/oracle/wookiee).
