Fetching Data From APIGEE and pushing into Mirth Via RabbitMQ

This will be a interesting blog post. In this post I will explain the following:

  1. Fetching the data from APIGEE (URL re-routing) in JAVA
  2. Pushing the fetched data from APIGEE and inserting to RabbitMQ queue.
  3. Pull the data from Rabbit MQ with Mirthconnect.

1. CREATE APIGEE URL:

APIGEE is available for both enterprise and normal non-payable version. You can sign into Apigee. There are various purposes and use-cases available to use Apigee but i’m currently using this Apigee as a URL re-routing mechanism.

Take any source of JSON data which can be available free. Sign in to APIGEE and click on API Proxies. In the opening page click on +PROXY button on the right hand side top. This will open a new page with the following information

apigee-1

Once the Above screen appears select the first option “REVERSE PROXY” this will act as a URL re-routing mechanism. You will have a actual URL but you will not use that URL for communicating with the clients instead, you will give one URL which which will be mapped to the original URL.

Click next on selecting the first option. Then you will see the below screen as shown:

apigee-2

In the above screen on Proxy Name you have to fill out the name that you wish to give as a proxy name in the first text box in my case I have provided (vibinces-eval-test). In Proxy Base Path  you need to provide a sub-context of your API name I have provided (apigeejsonprofile). In the Existing API you need to provide the full URL path of existing JSON API. Description is optional field, you can either provide it or not.

Once it is created my url looked like this http://vibinces-eval-test.apigee.net/apigeejsonprofile you might get a URL as well with the name of your choice. In the Security  tab, It is advised to select CORS headers on browse. Because it is always possible to get the cross-origin error when you try to access data from browsers which are not verified properly. Im also using no authorization for the API.

apigee-3

In the Next tab you can see how the data provided is converted to your URL. It is also fascinating that APIGEE provides you two types of URL that can be used both in Testing or BETA and as well as one for PROD.

apigee-4

Now your URL re-router is created. That is if  you hit the URL http://vibinces-eval-test.apigee.net/apigeejsonprofile you can see the JSON data that actually belongs to some other URL.

2. JAVA CODE – TO FETCH DATA FROM APIGEE:

I’m going to create the below two classes:

  1. FetchJsonFromApigee.java
  2. PushApigeeDataToRabbitMQ.java

1. FetchJsonFromApigee.java:

import java.io.BufferedReader;
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
public class FetchJsonFromApigee {
public static String call_me() throws Exception {
String url = “http://vibinces-eval-test.apigee.net/apigeejsonprofile”;
URL obj = new URL(url);
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
con.setRequestMethod(“GET”);
con.setRequestProperty(“User-Agent”, “Mozilla/5.0”);
int responseCode = con.getResponseCode();
System.out.println(“Response Code : ” + responseCode);
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in .readLine()) != null) {
response.append(inputLine);
} in .close();
System.out.println(“response : ” + response.toString());
return response.toString();
}
public String sendingMessage() throws Exception {
String pushedJsonMessage = FetchJsonFromApigee.call_me();
return pushedJsonMessage;
}
}

2. PushApigeeDataToRabbitMQ.java:

import java.io.IOException;
import java.util.concurrent.TimeoutException;

import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;

/**
* @author Vibinchander.V
*
*/
public class PushApigeeDataToRabbitMQ {
private final static String QUEUE_NAME = “TestQueuing”;

public static void passMessage(String message) throws IOException, TimeoutException {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost(“localhost”);
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
channel.basicPublish(“”, QUEUE_NAME, null, message.getBytes());
System.out.println(” [x] Sent ‘” + message + “‘”);
channel.close();
connection.close();
}

public static void main(String[] args) throws IOException, TimeoutException {
FetchJsonFromApigee getData = new FetchJsonFromApigee();
String passMessage = null;
try {
passMessage = getData.sendingMessage();
} catch (Exception e) {
e.printStackTrace();
}
PushApigeeDataToRabbitMQ.passMessage(passMessage);
System.out.println(“Executed Main Method !!!”);
}
}

When you run the first class you can see that the data is fetched from APIGEE and pushed into RabbitMQ message queue.

3. Write a JAR file that will pull data from RabbitMQ:

I’m writing the below class. this class will be used inside the Mirth tool which will act as a consumer to pull the data out of RabbitMQ.

import java.io.IOException;
import java.io.IOException;
import java.util.concurrent.TimeoutException;

import com.rabbitmq.client.AMQP;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
import com.rabbitmq.client.Consumer;
import com.rabbitmq.client.DefaultConsumer;
import com.rabbitmq.client.Envelope;
import com.rabbitmq.client.QueueingConsumer;
/**
 *
@author Vibinchander.V
 *
 */
public class QueueConsumer {
 public String returnMessage(String queueName) throws IOException, TimeoutException {
  ConnectionFactory factory = new ConnectionFactory(); 
factory.setHost(“localhost”); 
Connection connection = factory.newConnection(); 
Channel channel = connection.createChannel(); 
channel.queueDeclare(queueName, false, false, false, null); 
boolean noAck = false; 
@SuppressWarnings(“deprecation”) 
QueueingConsumer consumerVal = new QueueingConsumer(channel);  channel.basicConsume(queueName, noAck, consumerVal); 
boolean runInfinite = true; 
QueueingConsumer.Delivery delivery = null; 
//while (runInfinite) {  //QueueingConsumer.Delivery delivery; 

try {   
delivery = consumerVal.nextDelivery(); 
} catch (InterruptedException ie)
{   // continue; 
}  //System.out.println(“Message received” + new String(delivery.getBody()));  channel.basicAck(delivery.getEnvelope().getDeliveryTag(), false); 
//} 
channel.close(); 
connection.close();
  return new String(delivery.getBody()); 
}
}

Make the above program in a JAR file put this inside the custom lib folder of Mirth. you might have to include other JAR files as well. Inside the Mirth source connector Javascript reader write the below code:

var queueConsumer = new org.envision.queuing.QueueConsumer();
msg = queueConsumer.returnMessage(“TestQueuing”);
logger.debug(msg);
return msg;

When you run the first FetchDataFromApigee.java you can see that data will be fetched from Apigee and pushed to RabbitMQ queue and immediately pulled by Mirth consumer.

Happy Integrations!!!!!

 

Advertisements

Install Tomcat7 and RabbitMQ in Fedora or Amazon EC2 AMI

INSTALL TOMCAT 7:

1.sudo yum update  (Do a general update of your system)

2. sudo yum install tomcat7 tomcat7-webapps

3. sudo service tomcat7 start

4. cd /usr/share/tomcat7

Add .WAR files into the folder /usr/share/tomcat7

By Default Tomcat will take 8080 as a running port number.

CHECK : http://54.175.172.38:8080/

INSTALL RABBITMQ

1. RABBITMQ is built on top of Erlang langage. So we need to install that first.

yum install erlang

2. Get the RPM downloaded for RabbitMQ latest version here I have used (3.2.2)

sudo wget http://www.rabbitmq.com/releases/rabbitmq-server/v3.2.2/rabbitmq-server-3.2.2-1.noarch.rpm

3. Import rabbitmq-signing-key-public.asc for public key signing

sudo rpm –import http://www.rabbitmq.com/rabbitmq-signing-key-public.asc

4. Install the rpm downloaded

yum install rabbitmq-server-3.2.2-1.noarch.rpm

5. We have to auto-run RabbitMQ once your System/Server is ON to do that type the below command

sudo chkconfig rabbitmq-server on

6. Start the RabbitMQ Server.

sudo service rabbitmq-server start

7. Note: Upto this point. You can access your rabbitmq only via command prompt. If you want to enable the web console, then use the below command

sudo rabbitmq-plugins enable rabbitmq_management

After the above command is done. Your RabbitMQ will work by default on port 15672. Enable that port in your system.

Then restart or stop/start rabbitmq service again.

 

How to install and work with RabbitMQ – Part 1

This blog is about installation and working with one of the most useful message queuing system called RabbitMQ.

Before beginning, it is important for us to know what it actually does. This is more like a server mainly used for better queuing of messages. It uses a protocol called AMQP Advanced Message Queuing Protocol which enables the data to queue in a better rate and then process them.

This RabbitMQ is built on a language which is most infamous now and its called Erlang and it is built on Open Telecom Platform Framework for clustering and failover.

Previously there was a disadvantage in using this RabbitMQ because everything has to be handled via command prompt. People who uses this server were not tech savvy to do that. Later in their version they created a management studio that enables even a layman to work better.

It is also important for us to understand that not only RabbitMQ but also apache has a similar feature of server. Apache’s distribution of queuing mechanism is called Apache Kafka. We will discuss more about the both in fore coming days.

INSTALLATION OF RABBIT-MQ:

Step 1: Install Erlang –  We need to install Erlang and its corresponding components first before installing RabbitMQ. RabbitMQ will not work if Erlang is not installed. Download the latest version of Erlang from here.

Step 2: Once the installation is done, you have to set the environment config of ERLANG_HOME. If you are using Windows 10 you don’t need to setup the environment variable. It will be automatically setup as shown below.rabbitmq-Setup

Step 3: Install the latest version of RabbitMQ from here. This installation is pretty straight forward. If the installation of the Erlang is done without any problem then this will also work as smooth as butter.

Step 4: RabbitMQ runs by default as a windows service. We don’t need to explicitly invoke it. But at this stage you will have to do everything in the command line interface. Which is a tedious task. So it is better to work with the web management console interface of this.

  • Open command line interface with Admin access.
  • Navigate to C:\Program Files (x86)\RabbitMQ Server\rabbitmq_server-3.3.4\sbin 
  • Run the command rabbitmq-plugins.bat enable rabbitmq_management to enable the plugin of web management
  • Once, the above command is done, then re-install the RabbitMQ  using following commands:
  • rabbitmq-service.bat stop
    rabbitmq-service.bat install
    rabbitmq-service.bat start
  • Note: you can remain in the same directory to perform this.

rabbitmq-Setup1

If everything works fine as shown above in the screenshot. Then you can now open the web console of the RabbitMQ. you can access the RabbitMQ using the url http://localhost:15672/ the default username and password for this console will be guest guest

Will post more POC’s integrating RabbitMQ with Mirthconnect in future posts.

Happy Integrations!!!!!

 

 

 

Integrating – AWS EC2 (MySQL) to Mirth Engine

For this post. I have purchased a personal EC2 instance in AWS environment (a free tier for one year). Specifically an Amazon-AMI Image AWS instance with Fedora Operating System.

In the remote EC2 system, MySQL is deployed and a database is created as test. Once the DB is created in the EC2 instance, you have to create a table with some sample patient demographics information.

How to Access AWS remote server?

AWS_putty_access_1

Open the putty client and put the AWS amazon hostname in the Host Name or IP adress text box. Else you can put the elastic IP of your AWS instance there. It doesn’t always need to be a complete hostname.

  1. Then select SSH on the left side window of the putty and select on Auth. 
  2. Once you selected Auth on the right window pane click on the Browse button and select the private key you have downloaded from the amazon.
  3. This private key will be a .ppk file. This PPK file will be necessary to establish the SSH connectivity between your putty client and the AWS system.

Put_Private_Key_For_Access

  1. Once this is done click on the Open button on the bottom.
  2. Once you open you will be prompted to enter the username. If you purchased a Linux Ubuntu System the default user name will be ubuntu.
  3. If you have purchased a different system (say here I have purchased the Amazon AMI system). For the Amazon AMI system you need the password as ec2-user. 
  4. you can install MySQL linux distribution based the version you use. For Amazon AMI linux it uses the Fedora System. For Fedora use this command to install the MySQL. Before executing the below command do the command sudo yum update once.
dnf install mysql-community-server

For the Debian Ubuntu distribution use the below command for the MySQL installation

  • sudo apt-get update
  • sudo apt-get install mysql-server

Once installation is done. Log into your MySQL database on the AWS linux box. You can do this by typing the below command in the putty client box.

mysql -u root -p

Once it is done, you will enter into mysql> type show databases; initially you will not have a new database been created at your end. you have to create a new database for yourself. use the below command and create a new database.

create database test;

This will create a new database. Now we have to use this database and create the tables in it. Use the below command to use this database;

use test;

Here, test is the database name that i’m creating in the box. you can have the name of your choice. Then create a table with the following fields as provided in the screen shot below.

Mysql_Table_Structutre

  1. Once you have created it, log on to your AWS web console. And open the port in the security groups on the web console.
  2. Click on Edit button after selecting the security group and then add the port 3306 on TCP.
  3. Only when we do this the inbound socket of your remote system will open, then only your local Mirth system can establish the communication to the system in AWS environment.

In case if you install any application server like the Apache or Tomcat you would initially want to open the specific ports which are specified in the httpd.conf or catalina.conf files. We want to go to the security group on the AWS console and then open enable those inbound ports, then only your (IP+Port) combination will work in the browser, this IP+Port combination is technically referred as Socket.

Create a channel in your local Mirth Connect. make the source to the javascript reader and keep the polling frequency of your interest to fetch the data from the DB. In the source connector area provide the following code.

var dbConn;
// AWS Mysql Credentials
var mySqlDriver = “com.mysql.jdbc.Driver”;
// 54.175.172.38 – Server IP
// 3306 – Mysql Default Port number
// test – Database in Remote server
var mySqlConnectionTemplate = “jdbc:mysql://54.175.172.38:3306/test”;
var mySqlUserName = “root”;

// Create Parent Tag <PatientDemographics>
var patientDemographicsXml = new XML(‘<PatientDemographics></PatientDemographics>’);
// Create Parent for individual patient information <IndividualPatientInformation>
var individualPatientInfoXml = new XML(‘<IndividualPatientInformation></IndividualPatientInformation>’);

try {
// MySQL connection Template
dbConn = DatabaseConnectionFactory.createDatabaseConnection(mySqlDriver, mySqlConnectionTemplate, mySqlUserName, );
// Select statement
// patient_information – is the table name
result = dbConn.executeCachedQuery(“select * from patient_information“);

// Loop through the resultset value
while (result.next()) {
individualPatientInfoXml[‘PatientId’] = result.getInt(“pid”);
individualPatientInfoXml[‘PatientFirstName’] = result.getString(“patient_first_name”);
individualPatientInfoXml[‘PatientLastName’] = result.getString(“patient_last_name”);
individualPatientInfoXml[‘PatientMiddleName’] = result.getString(“patient_middle_ name”);
individualPatientInfoXml[‘PatientSuffixName’] = result.getString(“patient_suffix_name”);
individualPatientInfoXml[‘PatientDateOfBirth’] = result.getString(“patient_date_of_birth”);
individualPatientInfoXml[‘PatientGender’] = result.getString(“patient_gender”);
individualPatientInfoXml[‘PatientAge’] = result.getInt(“patient_age”);
individualPatientInfoXml[‘PatientAddress1’] = result.getString(“patient_address_1”);
individualPatientInfoXml[‘PatientAddress2’] = result.getString(“patient_address_2”);
individualPatientInfoXml[‘PatientEmailAddress’] = result.getString(“patient_emailAddress”);
individualPatientInfoXml[‘PatientTelecomNumber’] = result.getString(“patient_telecom_number”);
individualPatientInfoXml[‘PatientRace’] = result.getString(“patient_race”);
individualPatientInfoXml[‘PatientEthincity’] = result.getString(“patient_ethincity”);
individualPatientInfoXml[‘PatientMaritalStatus’] = result.getString(“patient_maritalstatus”);
individualPatientInfoXml[‘PatientLanguage’] = result.getString(“patient_language”);
individualPatientInfoXml[‘PatientCountry’] = result.getString(“patient_country”);
individualPatientInfoXml[‘PatientState’] = result.getString(“patient_state”);
individualPatientInfoXml[‘PatientCity’] = result.getString(“patient_city”);
individualPatientInfoXml[‘PatientZipCode’] = result.getString(“patient_zipcode”);
individualPatientInfoXml[‘PatientSSN’] = result.getString(“patient_ssn”);
individualPatientInfoXml[‘PatientDriverLicense’] = result.getString(“patient_driver_license”);

patientDemographicsXml[‘PatientDemographics’] += individualPatientInfoXml;

individualPatientInfoXml = new XML(‘<IndividualPatientInformation></IndividualPatientInformation>’);
}

msg = patientDemographicsXml;

return msg;

} finally {
if (dbConn) {
dbConn.close();
}
}

Once the connector code is created then you will be able to fetch all the data which are available in the database as a batch, instead of single row entries. You will be able to accumulate all the data from the database on each row as shown below:

Mysql_data_from_table

These data will be accumulated inside mirth as a XML data created in batch and the output of your mirth data will be as shown below:

<PatientDemographics>
<IndividualPatientInformation>
<PatientId>3301</PatientId>
<PatientFirstName>Jennifer</PatientFirstName>
<PatientLastName>Jaquline</PatientLastName>
<PatientMiddleName>Fernandez</PatientMiddleName>
<PatientSuffixName>III</PatientSuffixName>
<PatientDateOfBirth>1994-04-01</PatientDateOfBirth>
<PatientGender>F</PatientGender>
<PatientAge>23</PatientAge>
<PatientAddress1>No:8, washington, test drive</PatientAddress1>
<PatientAddress2>Oregan, detroit</PatientAddress2>
<PatientEmailAddress>jenn@test.com</PatientEmailAddress>
<PatientTelecomNumber>12356722</PatientTelecomNumber>
<PatientRace>2106-3</PatientRace>
<PatientEthincity>2131-5</PatientEthincity>
<PatientMaritalStatus>S</PatientMaritalStatus>
<PatientLanguage>EN</PatientLanguage>
<PatientCountry>US</PatientCountry>
<PatientState>GA</PatientState>
<PatientCity>oregan</PatientCity>
<PatientZipCode>55023</PatientZipCode>
<PatientSSN>123456789</PatientSSN>
<PatientDriverLicense>AXXW12345</PatientDriverLicense>
</IndividualPatientInformation>
<IndividualPatientInformation>
<PatientId>3302</PatientId>
<PatientFirstName>James</PatientFirstName>
<PatientLastName>Rupert</PatientLastName>
<PatientMiddleName>Jones</PatientMiddleName>
<PatientSuffixName>IV</PatientSuffixName>
<PatientDateOfBirth>1993-04-22</PatientDateOfBirth>
<PatientGender>M</PatientGender>
<PatientAge>24</PatientAge>
<PatientAddress1>4/12 Stevie Street, jj colony</PatientAddress1>
<PatientAddress2>Michigan, detroit</PatientAddress2>
<PatientEmailAddress>james@test.com</PatientEmailAddress>
<PatientTelecomNumber>12005678</PatientTelecomNumber>
<PatientRace>2028-9</PatientRace>
<PatientEthincity>2131-5</PatientEthincity>
<PatientMaritalStatus>M</PatientMaritalStatus>
<PatientLanguage>EN</PatientLanguage>
<PatientCountry>US</PatientCountry>
<PatientState>OR</PatientState>
<PatientCity>Oregan</PatientCity>
<PatientZipCode>44336</PatientZipCode>
<PatientSSN>987654321</PatientSSN>
<PatientDriverLicense>AXXE12309</PatientDriverLicense>
</IndividualPatientInformation>
</PatientDemographics>

Happy Integrations !!!!!

 

Blog at WordPress.com.

Up ↑