Archive for June, 2011

Turmeric In Action…

Posted: June 29, 2011 in Events, Turmeric

Turmeric with Cloud Mashups Webinars

Data services present a challenge to many businesses on the web as they are channeled from various partners. Turmeric, a proven open source solution in the eBay environment, helps you achieve seamless integration capabilities to address these data demands. It is a stable, fast, SOA runtime along with Eclipse-based service development tools, and web based management and reporting.

Teams at Intalio and eBay designed a working example to demonstrate Turmeric’s integration with Amazon services. Learn from these development gurus how this solution results in low latency messaging and protocol or format transformations.

Agenda:
Wed, Jun 29, 2011 9:00 PM – 10:00 PM ART
Thu, Jun 30, 2011 10:00 AM – 11:00 AM ART

Speakers:
Jose Alvarez Muguerza
Argentina

Manuel Chinea
Venezuela

Advertisements

Including Automated Tests and Automated Builds are some of the key aspects in a Continuous Integration development activity. Most of us are already familiar with these premises but, in the other hand, it is a common practice to commit/push our updates once all Unit Tests run successfully; leaving those Integration Tests and, why not, inter-dependencies troubles to be analyzed after the remote repo build. Anne Thomas Manes estimates that ongoing maintenance could achieve 92% of the total lifetime cost of an application.

Automated builds, with the use of any Integration Tool such as Hudson/Jenkins, are commonly performed “over there”, after our commits, but why don’t prevent to those possible dependency issues? or those failed integration tests?

Martin Fowler explains in his Continuous Integration article: “The point of testing is to flush out, under controlled conditions, any problem that the system will have in production. A significant part of this is the environment within which the production system will run. If you test in a different environment, every difference results in a risk that what happens under test won’t happen in production…” . Decreasing the gap between the Local dev environment and the Production environment reduces the chances of a Red Ball nightmare.

Continuing with my last post, eBayOpenSource with Sonatype Nexus, I decided to complete my dev environment with the use of Hudson+Sonar+Nexus tools.  Here are the simple steps:

Configure Sonatype nexus in your environment

– Go to Hudson home page and download the war file. You could install a native package as well, I prefer the war-way for OS upgrade purposes.

– Go to Sonar site, download latest zip file. v2.8 in this case.

Run Hudson with:

java -jar your_hudson_war

Hudson console runs on port 8080 by default. You can chage it by adding parameter –httpPort=8888

Go to Manage Hudson -> Manage Plugin and install Sonar plugin

In my case I use Git as repo, so I needed to add some extra plugins:

Then, go to Manage Hudon -> Configure System and add:

JAVA_HOME: in my case, /usr/lib/jvm/java-1.6.0-openjdk

GIT installations, give it a name

MAVEN_HOME: /home/jose/Tools/apache-maven-3.0

MAVEN_OPTS: -Xms512m -Xmx512m -XX:PermSize=128m -XX:MaxPermSize=128m

Sonar installations, give it a name

Create a new Hudson Job:

Add your Source Code Management:in my case GIT and  file:///home/jose/workspace/<any-git-project>

Branches to build: origin/master

Poll SCM: @hourly

and don’t forget to check last item, Post-build Actions: Sonar

press Save.

Now, it’s turn to run Sonar:

go to [Sonar_Home]/bin/[your_OS_Distro] and execute the sonar.sh script. It will load sonar service and console at localhost:9000

Go back to hudson console and press Build now in your Hudson Job….then magic happens!!!


Started by user anonymous
Checkout:workspace / /home/jose/.hudson/jobs/turmeric-policy/workspace - hudson.remoting.LocalChannel@3ed9c921
Using strategy: Default
Last Built Revision: Revision 2a40432f5ed44da0ada7e326b224198cb676f175 (origin/master)
Checkout:workspace / /home/jose/.hudson/jobs/turmeric-policy/workspace - hudson.remoting.LocalChannel@3ed9c921
Fetching changes from the remote Git repository
Fetching upstream changes from file:///home/jose/workspace-git/turmeric-policy
Commencing build of Revision 2a40432f5ed44da0ada7e326b224198cb676f175 (origin/master)
Checking out Revision 2a40432f5ed44da0ada7e326b224198cb676f175 (origin/master)
Found mavenVersion 3.0 from file jar:file:/home/jose/Tools/apache-maven-3.0/lib/maven-core-3.0.jar!/META-INF/maven/org.apache.maven/maven-core/pom.properties
Parsing POMs
downloaded artifact http://127.0.0.1:8081/content/repositories/ebayopensource-snapshots//org/ebayopensource/turmeric/policy/turmeric-policy-integration-tests-parent/1.0.0.2-SNAPSHOT/maven-metadata.xml
downloaded artifact http://127.0.0.1:8081/content/repositories/ebayopensource-snapshots//org/ebayopensource/turmeric/policy/turmeric-policy-parent/1.0.0.2-SNAPSHOT/maven-metadata.xml
[workspace] $ /usr/lib/jvm/java-1.6.0-openjdk/bin/java -Xms512m -Xmx512m -XX:PermSize=128m -XX:MaxPermSize=128m -cp /home/jose/.hudson/plugins/maven-plugin/WEB-INF/lib/maven3-agent-2.0.1.jar:/home/jose/Tools/apache-maven-3.0/boot/plexus-classworlds-2.2.3.jar org.jvnet.hudson.maven3.agent.Maven3Main /home/jose/Tools/apache-maven-3.0 /home/jose/.hudson/war/WEB-INF/lib/hudson-remoting-2.0.1.jar /home/jose/.hudson/plugins/maven-plugin/WEB-INF/lib/maven3-interceptor-2.0.1.jar 47965
���channel started
Executing Maven: -B -f /home/jose/.hudson/jobs/turmeric-policy/workspace/pom.xml sonar:sonar
[INFO] Scanning for projects...


You can see the “http://127.0.0.1:8081..." over there, it's Nexus in action....

Unfortunately I fell into the Duplicated Source Sonar issue. 😦

We should see something similar to:

see you in the next one…Hope SONAR-2444 be solved, otherwise I should find a workaround asap.

During last days I’ve suffered some Internet connection troubles, sometimes connections goes, other it turns very slow, similar at the 56kb modems in the early stages of internet. BTW, thanks to Movistar.

Those ones, can become (or not) in serious issues for an isolated developer or web page designer who could upload his advances once a day, or a week…but it turns critical for this developer who works together a multi-country team in an Open Source project.

For instance, in my daily labor I run maven builds, some days several of them…as you may know these maven builds look for and downloads huge quantities of files, artifacts, or dependencies…off course according your configurations, at least, I have to…….So, the need of an enhancement in this process arose me during these days. Does not take much time to decide one of the better and well-known tools that bring you the ability to manage the provisioning artifacts during development, mainly based on the its simply installation and configuration process. Welcome Nexus to my laptop. Nexus Open Source is available under the GNU Affero General Public License version 3 (AGPLv3): http://opensource.org/licenses/agpl-v3.html.

Is not the intention of this post to tell you what Nexus is or does, for that purpose just surf on the adequate and, for sure more complete;  nexus site. Instead of that, I wrote this lines to let you know how to get Nexus working on your computers.

So, firstly some key aspects about nexus:

  • Nexus is a Jetty unzip and run tool
  • You can manage your repositories from different approaches, in my case I set them as Proxy Repos: “When you proxy a remote repository with Nexus Open source you can control all aspects of the connection to a remote repository including security parameters, HTTP proxy settings. You can configure which mirrors Nexus will download artifacts from, and you can control how long Nexus will store artifacts and how it will expire artifacts which are no longer referenced by your build….”
  • Nexus provides a JavaService wrapper for Windows. Just in case u prefer MS W rather than any linux OS

… for a better feature description here you have a copy of the Sonatype Nexus Feature Matrix:

OS: Open Source version  –  Pro: Professional version

Then, fisrt of all, go to download page, choose the last one. In this case, Release 1.9.1.1. There you can choose it from a list of formats, war, tar, zip, etc…I download the nexus-oss-webapp-1.9.1.1-bundle.tar.gz

Decompress this file into a folder that you selected as nexus home. This package contains 2 folders inside: nexus-oss-webapp-1.9.1.1, the nexus distribution itself and the sonatype-work, empty for now, that is the folder where your local repo will resid. You can assign different /home/.. folders for each of them, or you can decompress it in just one. I choose that way, both, nexus-oss-webapp-1.9.1.1 and sonatype-work, under a /home/jose/.nexus/ folder.  In case you are damning the two-folder structure, just think in upgrading the nexus distribution in your environment, your local repo will remain intact!

Now, under your …./.nexus/nexus-oss-webapp-1.9.1.1/bin/jsw/ folder you can see several distribution folders. I use a linux over 64 bits, so I’m going to work with linux-x86-64. Be sure to run the appropriate according your OS.

Then, let’s create a linux link for nexus path:

ln -s .nexus/nexus-oss-webapp-1.9.1.1/bin/jsw/linux-x86-64/ nexus

You can run nexus as a service or just manually, but you need to remember to run it prior to start you development day. I preferred to set as service., and these were the steps:

  1. Copy either ${NEXUS_HOME}/bin/jsw/linux-ppc-64/nexus, ${NEXUS_HOME}/bin/jsw/linux-x86-32/nexus, or ${NEXUS_HOME}/bin/jsw/linux-x86-64/nexus to /etc/init.d/nexus.
  2. Make the /etc/init.d/nexus script executable – chmod 755 /etc/init.d/nexus
  3. Edit this script changing the following variables:
    • Change APP_NAME to “nexus”
    • Change APP_LONG_NAME to “Sonatype Nexus”
    • Add a variable NEXUS_HOME which points to your Nexus installation directory
    • Add a variable PLATFORM which contains either linux-x86-32, linux-x86-64, or linux-ppc-64
    • Change WRAPPER_CMD to ${NEXUS_HOME}/bin/jsw/${PLATFORM}/wrapper
    • Change WRAPPER_CONF to ${NEXUS_HOME}/bin/jsw/conf/wrapper.conf
    • Change PIDDIR to /var/run.
    • Add a JAVA_HOME variable which points to your local Java installation
    • Add a ${JAVA_HOME}/bin to the PATH

another useful setting is edit plexus.properties in ${NEXUS_HOME}/conf. You’ll see an element named webapp-context-path. Change this value from “/nexus” to “/”

cd /etc/init.d

update-rc.d nexus defaults

service nexus start

and finally…..

Starting Sonatype Nexus...

Sonatype Nexus will be running in your local and you can access to the admin console through you browser at. The defaults credentials are admin/admin123

So, once we have nexus already deployed we need to add some custom parameters and configuration in order to be useful during our maven deploys.

During these days I’m working with a project from eBayOpenSource community, so, let’s assign those repositories.

Basically for Turmeric we need to add two repositories:

Release repository:

and Snapshot Repository:

And, finally we need to update our .m2 settings file in order to run our maven build integrated with nexus. For a better HOW-TO explanation let me post my maven’s settings.xml file:

<?xml version="1.0" encoding="UTF-8"?>
<settings>
<!-- If you want to deploy content (SNAPSHOT or RELEASE) to the
Nexus instance at http://ph-0146.eva.ebay.com/nexus/
You'll need an account, and then you can enable this section.

To get an account, create an INFRA issue requesting the account.
Include your svn userid in the request.
<servers>
<server>
<id>ebayopensource-repo</id>
<username>{your.username}</username>
<password>{your.password}</password>
</server>
</servers>
-->

<!-- Search Order for Maven Plugins not configured in a project -->
<pluginGroups>
<pluginGroup>com.ebayopensource.plugins.maven</pluginGroup>
<pluginGroup>com.ebay.ebox.plugins</pluginGroup>
<pluginGroup>org.mortbay.jetty</pluginGroup>
<pluginGroup>org.codehaus.mojo</pluginGroup>
<pluginGroup>org.sonatype.maven.plugins</pluginGroup>
</pluginGroups>

<profiles>
<profile>
<id>turmeric</id>

<pluginRepositories>
<pluginRepository>
<id>ebayopensource-releases</id>
<name>eBayOpenSource Release Repository</name>
<url>http://127.0.0.1:8081/nexus/content/repositories/ebayopensource/</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</pluginRepository>
<pluginRepository>
<id>ebayopensource-snapshots</id>
<name>eBayOpenSource Snapshots Repository</name>
<url>http://127.0.0.1:8081/nexus/content/repositories/ebayopensource-snapshots/</url>
<releases>
<enabled>false</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
<updatePolicy>interval:30</updatePolicy>
<checksumPolicy>warn</checksumPolicy>
</snapshots>
</pluginRepository>
</pluginRepositories>

<repositories>
<repository>
<id>ebayopensource-releases</id>
<name>eBayOpenSource Release Repository</name>
<url>http://127.0.0.1:8081/nexus/content/repositories/ebayopensource/</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
<repository>
<id>ebayopensource-snapshots</id>
<name>eBayOpenSource Snapshots Repository</name>
<url>http://127.0.0.1:8081/nexus/content/repositories/ebayopensource-snapshots/</url>
<releases>
<enabled>false</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
<updatePolicy>interval:30</updatePolicy>
<checksumPolicy>warn</checksumPolicy>
</snapshots>
</repository>
</repositories>
</profile>
</profiles>
<activeProfiles>
<activeProfile>turmeric</activeProfile>
</activeProfiles>

<mirrors>
<mirror>
<id>Nexus</id>
<mirrorOf>central,eviware</mirrorOf>
<name>Nexus Public Mirror.</name>
<url>http://127.0.0.1:8081/nexus/content/groups/public</url>

</mirrors>

</settings>

Take care about the repositories id that you set, ie: ebayopensource-snapshots. These need to be the same as you defined in Nexus repo.

And….that’s all!!! now let’s do a maven build.
mvn clean package

During this process maven should attempt to download all your artifacts from http://localhost:8081&#8230;.. Of course first time you run your build, your nexus repo is empty so, the consuming time will not be decreased, but, try with your second build.

NOTE: If after your first maven build, maven is still getting some artifacts from any other external repository just review your Nexus Repositories looking for a wrong setup or just for a missing custom repository.

now, run your builds and enjoy your new Free Time.