Fetching Data From APIGEE and pushing into Mirth Via RabbitMQ

This will be a interesting blog post. In this post I will explain the following:

  1. Fetching the data from APIGEE (URL re-routing) in JAVA
  2. Pushing the fetched data from APIGEE and inserting to RabbitMQ queue.
  3. Pull the data from Rabbit MQ with Mirthconnect.

1. CREATE APIGEE URL:

APIGEE is available for both enterprise and normal non-payable version. You can sign into Apigee. There are various purposes and use-cases available to use Apigee but i’m currently using this Apigee as a URL re-routing mechanism.

Take any source of JSON data which can be available free. Sign in to APIGEE and click on API Proxies. In the opening page click on +PROXY button on the right hand side top. This will open a new page with the following information

apigee-1

Once the Above screen appears select the first option “REVERSE PROXY” this will act as a URL re-routing mechanism. You will have a actual URL but you will not use that URL for communicating with the clients instead, you will give one URL which which will be mapped to the original URL.

Click next on selecting the first option. Then you will see the below screen as shown:

apigee-2

In the above screen on Proxy Name you have to fill out the name that you wish to give as a proxy name in the first text box in my case I have provided (vibinces-eval-test). In Proxy Base Path  you need to provide a sub-context of your API name I have provided (apigeejsonprofile). In the Existing API you need to provide the full URL path of existing JSON API. Description is optional field, you can either provide it or not.

Once it is created my url looked like this http://vibinces-eval-test.apigee.net/apigeejsonprofile you might get a URL as well with the name of your choice. In the Security  tab, It is advised to select CORS headers on browse. Because it is always possible to get the cross-origin error when you try to access data from browsers which are not verified properly. Im also using no authorization for the API.

apigee-3

In the Next tab you can see how the data provided is converted to your URL. It is also fascinating that APIGEE provides you two types of URL that can be used both in Testing or BETA and as well as one for PROD.

apigee-4

Now your URL re-router is created. That is if  you hit the URL http://vibinces-eval-test.apigee.net/apigeejsonprofile you can see the JSON data that actually belongs to some other URL.

2. JAVA CODE – TO FETCH DATA FROM APIGEE:

I’m going to create the below two classes:

  1. FetchJsonFromApigee.java
  2. PushApigeeDataToRabbitMQ.java

1. FetchJsonFromApigee.java:

import java.io.BufferedReader;
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
public class FetchJsonFromApigee {
public static String call_me() throws Exception {
String url = “http://vibinces-eval-test.apigee.net/apigeejsonprofile”;
URL obj = new URL(url);
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
con.setRequestMethod(“GET”);
con.setRequestProperty(“User-Agent”, “Mozilla/5.0”);
int responseCode = con.getResponseCode();
System.out.println(“Response Code : ” + responseCode);
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in .readLine()) != null) {
response.append(inputLine);
} in .close();
System.out.println(“response : ” + response.toString());
return response.toString();
}
public String sendingMessage() throws Exception {
String pushedJsonMessage = FetchJsonFromApigee.call_me();
return pushedJsonMessage;
}
}

2. PushApigeeDataToRabbitMQ.java:

import java.io.IOException;
import java.util.concurrent.TimeoutException;

import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;

/**
* @author Vibinchander.V
*
*/
public class PushApigeeDataToRabbitMQ {
private final static String QUEUE_NAME = “TestQueuing”;

public static void passMessage(String message) throws IOException, TimeoutException {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost(“localhost”);
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
channel.basicPublish(“”, QUEUE_NAME, null, message.getBytes());
System.out.println(” [x] Sent ‘” + message + “‘”);
channel.close();
connection.close();
}

public static void main(String[] args) throws IOException, TimeoutException {
FetchJsonFromApigee getData = new FetchJsonFromApigee();
String passMessage = null;
try {
passMessage = getData.sendingMessage();
} catch (Exception e) {
e.printStackTrace();
}
PushApigeeDataToRabbitMQ.passMessage(passMessage);
System.out.println(“Executed Main Method !!!”);
}
}

When you run the first class you can see that the data is fetched from APIGEE and pushed into RabbitMQ message queue.

3. Write a JAR file that will pull data from RabbitMQ:

I’m writing the below class. this class will be used inside the Mirth tool which will act as a consumer to pull the data out of RabbitMQ.

import java.io.IOException;
import java.io.IOException;
import java.util.concurrent.TimeoutException;

import com.rabbitmq.client.AMQP;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
import com.rabbitmq.client.Consumer;
import com.rabbitmq.client.DefaultConsumer;
import com.rabbitmq.client.Envelope;
import com.rabbitmq.client.QueueingConsumer;
/**
 *
@author Vibinchander.V
 *
 */
public class QueueConsumer {
 public String returnMessage(String queueName) throws IOException, TimeoutException {
  ConnectionFactory factory = new ConnectionFactory(); 
factory.setHost(“localhost”); 
Connection connection = factory.newConnection(); 
Channel channel = connection.createChannel(); 
channel.queueDeclare(queueName, false, false, false, null); 
boolean noAck = false; 
@SuppressWarnings(“deprecation”) 
QueueingConsumer consumerVal = new QueueingConsumer(channel);  channel.basicConsume(queueName, noAck, consumerVal); 
boolean runInfinite = true; 
QueueingConsumer.Delivery delivery = null; 
//while (runInfinite) {  //QueueingConsumer.Delivery delivery; 

try {   
delivery = consumerVal.nextDelivery(); 
} catch (InterruptedException ie)
{   // continue; 
}  //System.out.println(“Message received” + new String(delivery.getBody()));  channel.basicAck(delivery.getEnvelope().getDeliveryTag(), false); 
//} 
channel.close(); 
connection.close();
  return new String(delivery.getBody()); 
}
}

Make the above program in a JAR file put this inside the custom lib folder of Mirth. you might have to include other JAR files as well. Inside the Mirth source connector Javascript reader write the below code:

var queueConsumer = new org.envision.queuing.QueueConsumer();
msg = queueConsumer.returnMessage(“TestQueuing”);
logger.debug(msg);
return msg;

When you run the first FetchDataFromApigee.java you can see that data will be fetched from Apigee and pushed to RabbitMQ queue and immediately pulled by Mirth consumer.

Happy Integrations!!!!!

 

Advertisements

Install Tomcat7 and RabbitMQ in Fedora or Amazon EC2 AMI

INSTALL TOMCAT 7:

1.sudo yum update  (Do a general update of your system)

2. sudo yum install tomcat7 tomcat7-webapps

3. sudo service tomcat7 start

4. cd /usr/share/tomcat7

Add .WAR files into the folder /usr/share/tomcat7

By Default Tomcat will take 8080 as a running port number.

CHECK : http://54.175.172.38:8080/

INSTALL RABBITMQ

1. RABBITMQ is built on top of Erlang langage. So we need to install that first.

yum install erlang

2. Get the RPM downloaded for RabbitMQ latest version here I have used (3.2.2)

sudo wget http://www.rabbitmq.com/releases/rabbitmq-server/v3.2.2/rabbitmq-server-3.2.2-1.noarch.rpm

3. Import rabbitmq-signing-key-public.asc for public key signing

sudo rpm –import http://www.rabbitmq.com/rabbitmq-signing-key-public.asc

4. Install the rpm downloaded

yum install rabbitmq-server-3.2.2-1.noarch.rpm

5. We have to auto-run RabbitMQ once your System/Server is ON to do that type the below command

sudo chkconfig rabbitmq-server on

6. Start the RabbitMQ Server.

sudo service rabbitmq-server start

7. Note: Upto this point. You can access your rabbitmq only via command prompt. If you want to enable the web console, then use the below command

sudo rabbitmq-plugins enable rabbitmq_management

After the above command is done. Your RabbitMQ will work by default on port 15672. Enable that port in your system.

Then restart or stop/start rabbitmq service again.

 

How to install and work with RabbitMQ – Part 1

This blog is about installation and working with one of the most useful message queuing system called RabbitMQ.

Before beginning, it is important for us to know what it actually does. This is more like a server mainly used for better queuing of messages. It uses a protocol called AMQP Advanced Message Queuing Protocol which enables the data to queue in a better rate and then process them.

This RabbitMQ is built on a language which is most infamous now and its called Erlang and it is built on Open Telecom Platform Framework for clustering and failover.

Previously there was a disadvantage in using this RabbitMQ because everything has to be handled via command prompt. People who uses this server were not tech savvy to do that. Later in their version they created a management studio that enables even a layman to work better.

It is also important for us to understand that not only RabbitMQ but also apache has a similar feature of server. Apache’s distribution of queuing mechanism is called Apache Kafka. We will discuss more about the both in fore coming days.

INSTALLATION OF RABBIT-MQ:

Step 1: Install Erlang –  We need to install Erlang and its corresponding components first before installing RabbitMQ. RabbitMQ will not work if Erlang is not installed. Download the latest version of Erlang from here.

Step 2: Once the installation is done, you have to set the environment config of ERLANG_HOME. If you are using Windows 10 you don’t need to setup the environment variable. It will be automatically setup as shown below.rabbitmq-Setup

Step 3: Install the latest version of RabbitMQ from here. This installation is pretty straight forward. If the installation of the Erlang is done without any problem then this will also work as smooth as butter.

Step 4: RabbitMQ runs by default as a windows service. We don’t need to explicitly invoke it. But at this stage you will have to do everything in the command line interface. Which is a tedious task. So it is better to work with the web management console interface of this.

  • Open command line interface with Admin access.
  • Navigate to C:\Program Files (x86)\RabbitMQ Server\rabbitmq_server-3.3.4\sbin 
  • Run the command rabbitmq-plugins.bat enable rabbitmq_management to enable the plugin of web management
  • Once, the above command is done, then re-install the RabbitMQ  using following commands:
  • rabbitmq-service.bat stop
    rabbitmq-service.bat install
    rabbitmq-service.bat start
  • Note: you can remain in the same directory to perform this.

rabbitmq-Setup1

If everything works fine as shown above in the screenshot. Then you can now open the web console of the RabbitMQ. you can access the RabbitMQ using the url http://localhost:15672/ the default username and password for this console will be guest guest

Will post more POC’s integrating RabbitMQ with Mirthconnect in future posts.

Happy Integrations!!!!!

 

 

 

Blog at WordPress.com.

Up ↑