Archive for the ‘Turmeric’ Category

It’s been a while since I posted my last note over here. It is a year with changes in my work life. As you may know Ebay has decided to minimize the effort dedicated to its open source platform, at least to TurmericSOA. As a consequence, the permanent staff they had assigned to TurmericSOA were extremely downsized to 1 resource.

Although Turmeric was an open source project, and as most of the Open Source initiatives, the developers were salaried, and full time dedicated. Due to the above taken decision, the coworkers in this project , me included, have bound in looking for new horizons.

TurmericSOA is still alive, is there….in Git repo, in forums, in IRC channel, all these resources are still alive, in fact, I do my humble contributions from time to time.
Some months ago I read an article from my friend Dave talking about the rod for measuring success. I totally agree with dave that success is not popularity neither viceversa, but we can’t avoid that outside doors work is a critical leverage, letting know the industry what you can offer them, namely the project broadcast strategy. That strategy takes its time,  depending on what your product is, this “time” could extend from one month to several years, in particular if we’re dealing with open source projects. If we only consider the TurmericSOA’s “broadcast strategy time”, in my opinion is was to early to leave that project adrift the open source community, but who knows, big company decisions have been always a Pandora’s box, and naturally, there must be some upper management decision that we’ll never know.
I honestly wish the best to TurmericSOA project and off course to each of whom collaborate with us. People from Jetty, Intalio and eBay itself, WSO2, MuleSoft guys, in particular to Manuel Chinea, Dave “kingargyle” Carver, Adam Lieber, Tiago Espinha, Jan Bartel, Michael Govoroy and Sastry Mallady

see you soon….

ebayOpenSource FLOSS show

Posted: February 28, 2012 in eBayOpenSource, Turmeric, VJET

“Netcasts you love from people you trust”, the FLOSS site bring us a new talk with our colleague Sastry Mallady, the eBay’s Technical Architect. Sastry comments about the ebayopensource and also mentions the 3 “open sourced” projects, Turmeric, ql.io and VJET.

Enjoy this weekly episode http://twit.tv/show/floss-weekly/201

c u!

RESTful requests with TurmericSOA

Posted: December 12, 2011 in REST, SOA, Turmeric

This time I’m gonna write about how to play in Turmeric with RESTful requests.

First of all we need a deployed service, any, in our Web Server, in my case Jetty 7, indeed, I use jetty-turmeric , this is a customized Jetty for running our Turmeric engine and deploying services in one or two simple steps…basically for simple but comprehensive tests.

The service I use is a sample EchoService, already included under the Turmeric Runtime module. So, under our WebServer/webapp folder we have a folder called example-echoservice. example-echoservice


|----META-INF
|----WEB-INF
        |-----classes
        |-----lib
               |-----ExampleEchoServiceV1-1.0.1.0-SNAPSHOT.jar
               |-----ExampleEchoServiceV1Impl-1.0.1.0-SNAPSHOT.jar
        |-----web.xml
|----index.html

Our web.xml deployment file should look like this:


<web-app
xmlns="http://java.sun.com/xml/ns/j2ee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/web-app_2_5.xsd"
metadata-complete="false"
version="2.5">
<display-name>Spfservlet Test</display-name>
<servlet>
<description>ExampleEchoServiceV1</description>
<display-name>ExampleEchoServiceV1</display-name>
<servlet-name>ExampleEchoServiceV1</servlet-name>
<servlet-class>org.ebayopensource.turmeric.runtime.spf.pipeline.SPFServlet</servlet-class>
<load-on-startup>1</load-on-startup>
<init-param>
<param-name>SOA_SERVICE_NAME</param-name>
<param-value>ExampleEchoServiceV1</param-value>
</init-param>
</servlet>
<servlet-mapping>
<servlet-name>ExampleEchoServiceV1</servlet-name>
<url-pattern>/ExampleEchoServiceV1</url-pattern>
</servlet-mapping>
</web-app>

The included Echo sample is ready to be deployed, so after copy/paste this project under our jetty-turmeric, the service could be achieved through: http://localhost:8080/example-echoservice/ExampleEchoServiceV1?echo&echoText=Hello and the response should be similar as:

<echoResponse>
<ack>Success</ack>
<output>HelloGuest</output>
</echoResponse>

Now, let’s review the main config file in this scenario, that is the ServiceConfig.xml under the ExampleEchoServiceV1Impl.jar By default it comes with a very simple an basic turmeric configuration:


<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<service-config xmlns="http://www.ebayopensource.org/turmeric/common/config">
<service-impl-class-name>org.ebayopensource.turmeric.example.v1.services.echoservice.impl.ExampleEchoServiceV1Impl</service-impl-class-name>
<service-interface-class-name>org.ebayopensource.turmeric.example.v1.services.echoservice.ExampleEchoServiceV1</service-interface-class-name>
<service-instance-config>
<provider-options>
<header-mapping-options>
<option name="X-TURMERIC-OPERATION-NAME">queryop</option>
<option name="X-TURMERIC-RESPONSE-DATA-FORMAT">query[format]</option>
</header-mapping-options>
</provider-options>

<protocol-processor version="1.1" name="SOAP11">
<indicator>
<transport-header name="X-TURMERIC-MESSAGE-PROTOCOL">SOAP11</transport-header>
</indicator>
<class-name>org.ebayopensource.turmeric.runtime.spf.impl.protocolprocessor.soap.ServerSOAPProtocolProcessor</class-name>
</protocol-processor>
<protocol-processor version="1.2" name="SOAP12">
<indicator>
<transport-header name="X-TURMERIC-MESSAGE-PROTOCOL">SOAP12</transport-header>
</indicator>
<class-name>org.ebayopensource.turmeric.runtime.spf.impl.protocolprocessor.soap.ServerSOAPProtocolProcessor</class-name>
</protocol-processor>
</service-instance-config>
</service-config>
<pre>

We have three main tag blocks here, lets forget the protocol-processor for now, they are SOAP related protocols, not usefull in our REST scenario…

Then, according this Turmeric wiki page, there some required parameters when we make a service call, but if we recall our endpoint (http://localhost:8080/example-echoservice/ExampleEchoServiceV1?echo&echoText=Hello) seems we forgot the X-TURMERIC-OPERATION-NAME. That is the purpose of queryop value in header-mapping-options, it automatically retrieves the first request parameter as the operation name, so we could replace an original and longer URL request http://localhost:8080/example-echoservice/ExampleEchoServiceV1?X-TURMERIC-OPERATION-NAME=echo&echoText=Hello by http://localhost:8080/example-echoservice/ExampleEchoServiceV1?echo&echoText=Hello.

In addition, one of our followers wants to achieve the service through this endpoint instead: http://localhost:8080/example-echoservice/ExampleEchoServiceV1/echo/Hello                <— pay attention, there is no question mark to specify the parameters

So, there are two simple steps you want to perform, first of all you need to update your deployment descriptor file for a propper intrepretation of your new URL, that is adding a wild card in your Servlet Mapping:

<servlet-mapping>
<servlet-name>ExampleEchoServiceV1</servlet-name>
<url-pattern>/ExampleEchoServiceV1/*</url-pattern>
</servlet-mapping>

Next step is to update the ServiceConfig.xml in order to read the latest word in our URL word as the echoText parameter, this tag section must be added following the header-mapping-options


<request-params-mapping>
<operation name="echo">
<option name="echoText">path[3]</option>
</operation>
</request-params-mapping>

That will read the 3rd expression in our URL

Please surf here for more info about URL mapping under REST request.

Also you can specify a pipeline process into the ServiceConfig.xml to provide Authentication, Rate Limiter capabilities or maybe custom handler to improve your service governance, but this  deserves another post 🙂

Hi dude..

In this post I’m going to show you a brief demo about the use of Rate Limiter feature with Turmeric SOA and Cassandra Integration.

Firstly let me explain you the scenario and the environment setup.

Scenario:

-A resource, a Service in this case,  must be denied after a member of a Subject Group makes 4 calls to an Echo Service

Environment:

  • A simple Echo Web Service deployed in two customised Jetty server, these are Jetty-Turmeric 1 and 2 (see chart Environment Setup)
  • Then Rate Limiter Service and Policy Service and PolicyAdmiUI deployed in other two Jetty servers
  • 4 Cassandra nodes running as stand alone services. In my case I used cassandra v 0.8.2
  • a turmericdb schema in a MySql database to store the predefined policies
  • any web browser, as REST client, FF in my case

Jetty-Turmeric Web Server Setup:

Download and decompress jetty-turmeric. This customised Jetty already provides a Echo Service as example. We need 4 instances (copies) of it, so let’s call it jetty-turmeric-1.0.1.0-SNAPSHOT-1, jetty-turmeric-1.0.1.0-SNAPSHOT-2, jetty-turmeric-1.0.1.0-SNAPSHOT-3 and jetty-turmeric-1.0.1.0-SNAPSHOT-4. Don’t forget to change their listening ports, otherwise you will get a nightmare of conflicts. I choose 8080, 8081, 8082 and 8083. Also you need to change SSL ports, and debugging port in case you want to do a remote debuggin with Eclipse:  -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8000

Then, lets prepare the Jetty #1 and #2, those who has the Echo Service Consumer. One of the yellow boxes we need to setup in the SIF is the AuthenticationHandler, that is a handler which makes call to the Authentication Service deployed in Jetty Server #3 and 4 respectively.  So we need to edit the {JETTY-HOME}/resources/META-INF/security/config/AuthenticationPolicy.xml file and add the operation against we need the Authentication process be run  :

<resource name="ExampleEchoServiceV1" default-authentication-method="basic" type="SERVICE">
<operation name="echo">
<authentication-method>basic</authentication-method>
</operation>
</resource>

Now, since we use our Turmeric-Cassandra-util package we need to add some jar dependencies under {JETTY-HOME}/lib/turmericsoa-security/ folder:
log4j-1.2.9.jar
TurmericUtils-0.9.0-Beta.jar

As some services are deployed under another Web Server, we must indicate the actual endpoint and don’t forget to change the Transport Protocol, from LOCAL to HTTP. This applies for {JETTY-HOME}/lib/turmericsoa-security/AuthenticationServiceConsumer-1.x.x.jar/META-INF/soa/client/config/AuthenticationService/ClientConfig.xml

<client-config-list xmlns="http://www.ebayopensource.org/turmeric/common/config"></pre>
<client-config service-name="{http://www.ebayopensource.org/turmeric/services}AuthenticationService">
<service-interface-class-name>org.ebayopensource.turmeric.services.authenticationservice.intf.AuthenticationService</service-interface-class-name>
<service-location>http://localhost:8082/security/AuthenticationServiceV1</service-location>
<client-instance-config>
<invocation-options>
<preferred-transport name="HTTP11">
<override-options>
<skip-serialization>true</skip-serialization>
</override-options>
</preferred-transport>
<request-data-binding>XML</request-data-binding>
<response-data-binding>XML</response-data-binding>
</invocation-options>
</client-instance-config>
</client-config>
</client-config-list>
<pre>

localhost:8083 for Jetty#2 setup

Also applies for {JETTY-HOME}/lib/turmericsoa-security/RateLimiterServiceConsumer-1.x.x.jar/META-INF/soa/client/config/RateLimiterService/ClientConfig.xml

Endpoint: http://localhost:8082/security/RateLimiterServiceV1 or http://localhost:8083/security/RateLimiterServiceV1 for Jetty#4

Now, lets prepare the Jetty #3 and #4.

As we can see in the above chart in Jetty#3 resides, Security Service, Policy Service and PolicyAdminUI (the Web UI for
policy Administration).
Download and extract them under webApps folder. Then the custom updates are:
For Security Service: None
For Policy Service: None
For PolicyAdminUI: in {Jetty3-HOME}/webapps/policy/lib/web.xml update the endpoint ports as Jetty3 has, that is 8082
for http and 8445 for shttp.

Later, in both {JETTY3-HOME} and {JETTY4-HOME} /resources/META-INF/config/cassandra.properties you need to add Rate
Limiter Keyspace info and also indicate there that cassandra will run in standalone mode instead of embedded mode,
default mode.

################################
###### RATE LIMITER COUNTER ####
################################
cassandra-rl-cluster-name=TestCluster
cassandra-host-ip=127.0.0.1
cassandra-rpc-port=9160
cassandra-rl-keyspace=rl

#column families
cassandra-active-rl-column-family=activeRL
cassandra-active-effect-column-family=activeEffect

embedded=false

This config should follow the cassandra configuration. Other cassandra files are not necessary to be changed due to we set embedded=false. This, tells Turmeric that Cassandra service is running in a standalone mode; setting to true TurmericSOA will start its own embedded Cassandra service.

If you need more datails on how to enable Rate Limiting to your own service surf on Turmeric Rate Limiting Setup page

enjoy it!!

Googleplex 2011

Posted: November 23, 2011 in eBayOpenSource, Eclipse, Turmeric, VJET

A new “googol” event is coming son, Nov 30th, the GooglePlex takes place at Olympus Mons Tech Talk Room,  Building CL4 in Mountain View, CA.

“Googleplex is a 1-day event for developers to learn about different Eclipse projects and related technologies. You are invited to attend and listen as experts from the Eclipse projects and Google share their experiences of using Eclipse. It’s also a great opportunity for you to meet and network with other Eclipse enthusiasts”

There you can learn about two projects under the EbayOpenSource ecosystem, VJET, an Eclipse plugin for javascript development by Justin Early (eBay) and TurmericSOA, the open source version of eBay’s SOA engine, by Dave Carver (Intalio)

Another interesting talks:
NASA Ames uses Eclipse RCP
Xtext success story: Google’s protobuf-dt
EGit_and_JGit_New_and_Noteworthy

Enjoy the devs party 🙂

 

 

Rule Syntax in TurmericSOA

Posted: November 10, 2011 in Turmeric, XACML

Hi,

Turmeric‘s next release will provide two new rule patterns for Rate Limiting Policies, which allow to limit calls for any XACML Subject or Subject Group. With them,  grows the rules flexibility as well as its throwput.

Recalling those years developing a Motohealth protocol, I cleaned up the dust from my writing capabilities in EBNF  🙂 , then, this is how the Rule syntax looks:

rule = opt-ws expression opt-ws | opt-ws expression opt-ws '||' opt-ws expression opt-ws | opt-ws expression opt-ws '&&' opt-ws expression
expression = global_hits | subject_hits | count

global_hits = 'HITS' opt-ws comparator opt-ws number
subject_hits = subject ':hits' opt-ws comparator opt-ws number
count = service [':' operation  ['.SubjectGroup' ['.Subject'] ] ] '.count' opt-ws comparator opt-ws number

comparator =  '<'|'>'|'<='|'=<'|'>='|'=>'|'=='
number =  {digit}
digit = '0'|'1'|'2'|'3'|'4'|'5'|'6'|'7'|'8'|'9'
character = lowercase-char | uppercase-char | digit | special-char
lowercase-char = "a" | "b" | "..." | "z"
uppercase-char = "A" | "B" | "..." | "Z"
special-char = '_'|'-'

subject = ip | dev | user | app
ip = range '.' range '.' range '.' range
range = ['0'|'1'|'2'] [digit] digit
dev = word
user = word
app = word

word = lowercase-char  {character | digit} | uppercase-char  {character | digit}

service = word
operation = word
opt-ws = [{' '}]

With this syntax we assume following are valid rules:

128.10.10.4:hits>1000: Limit that IP after 1000 calls*
HITS>10000: Limit any call after 10000 calls, regardless what and who made them*
MyService:my_operation.count>150: Limit any call to my_operation after 150 calls, regardless who made them*
MyService.count>100: Limit any call to myService after 100 calls, regardless the caller and the operation*

new ones….
MyService:my_operation.SubjectGroup.count>500: Limit any call to my_operation after 500 calls, made by a Subject Group*
MyService:my_operation.SubjectGroup.Subject.count>500
: Limit any call to my_operation after 500 calls, made by each Subject member of a Subject Group*

(*) Limiting action acts based on on the specified Effect action field.

Don’t forget Subject and SubjectGroup must be targets on the RL Policy definition. (FMI refer the Turmeric 1.0.0 wiki)

c u  soon….


Hi, I’m back. It has been a long time I don’t have the chance to write some lines over here. It has been a pretty working hard month focused in last release, btw, it is coming in next days.  Today I’m writing these few lines to invite you to the next Turmeric SOA show. I’ll will be there, showing an introduction of TurmericSOA to my Argentinean mates. Yes, this time Turmeric lands in Argentina. November 1st, at the Argentinean Java User Group annual event at ITBA, Buenos Aires.

Will be some other interesting talks during the event, such us “Challenges creating a iPaaS,scalability in MongoDB , pros & cons around ElasticSearch/Fume”  by Dan Diephouse from MuleSoft; “Uses and guidelines around Kanban development method” by Paulo Caroli from Thoughtworks.

I hope you have the chance to participate in this annual event. For more information surf in the Argentinean Java User Group home page . Also if you are interested in participate as presenter/attendee, do not hesitate to contact me

See you soon!!

New coming features…

Posted: September 14, 2011 in Cassandra, Turmeric

Here are some of the new features delivered within the next rlse.

      • Expanded Rest Support: SOA Framework will now support REST operations that come in with HTTP verbs PUT and DELETE in addition to GET and POST. Operation mapping in service config xml can be done for each http request type (GET, POST, PUT and DELETE).
      • OSGI – Now SOA consumers can use SOA in an OSGI safe way : Currently, SOA runtime loads resources and handlers from classloader. When running in OSGi environment, resources such as ClientConfig.xml, ServiceConfig.xml, and application customized handlers don’t belong the SOA runtime bundle. Now SOA OSGI fwk provides registration API which supplies directly streamed config resource information so that getResourceAsStream() does not need to be called.
      • Zero Config Consumer: This feature eliminates the need for separate consumer jar. Now Plugin has Simple and Advanced mode for Consumer creation from WSDL. In simple mode, consumer project would not be created. Runtime will automatically switch to use the default Client Config file.
      • Protocol Buffers Support: SOA fwk now supports the Google’s data format – Protocol Buffers along with existing data formats XML, FAST_INFOSET, JSON and NV. SOA tooling generates the required artifacts to handle protobuf if the service is enabled for protobuf. The developer no need to write any special code to use the protobuf format except the configuration in Client Config file just like any other format.
      • Distributed Rate Limiter counters:   With the use of Cassandra ring the rate limiter feature can be deployed in cluster mode
      • Aggregation data for Monitoring:   the powerful monitoring console will now show summarized data. It can also read data from distributed nodes in Cassandra.

									

Hi, when coding under Eclipse IDE v3.5 or above you can’t miss this extra for Turmeric development. It bring us some aditional support for the eBay Open Source Turmeric SOA project’s configuration files. Here we have a screenshot:

The project is looking for contributions, please feel free to fork the project on github

Turmeric In Action…

Posted: June 29, 2011 in Events, Turmeric

Turmeric with Cloud Mashups Webinars

Data services present a challenge to many businesses on the web as they are channeled from various partners. Turmeric, a proven open source solution in the eBay environment, helps you achieve seamless integration capabilities to address these data demands. It is a stable, fast, SOA runtime along with Eclipse-based service development tools, and web based management and reporting.

Teams at Intalio and eBay designed a working example to demonstrate Turmeric’s integration with Amazon services. Learn from these development gurus how this solution results in low latency messaging and protocol or format transformations.

Agenda:
Wed, Jun 29, 2011 9:00 PM – 10:00 PM ART
Thu, Jun 30, 2011 10:00 AM – 11:00 AM ART

Speakers:
Jose Alvarez Muguerza
Argentina

Manuel Chinea
Venezuela