Wednesday, September 11, 2013

Java keytool quick tips

Creating a keystore from an SSL key and cert

To store an SSL key for Jetty or other Java based webserver, you usually use keytool. You'll first need to convert the key to pkcs12 (we're calling the output file jetty.pkcs12) format like so (we're assuming your key is generated with openssl or something similar - also, we're in the same directory as the keys):

Convert mykey,key and mykey.crt to a pkcs12  format using openssl
( I highly recommend using a passphrase on the key)


openssl pkcs12 -inkey ./mykey.key -in ./mykey.crt -export -out ./jetty.pkcs12

Let's take that pkcs12 key and convert it into a Java keystore (we're calling the keystore mykeystore)




keytool -importkeystore -srckeystore jetty.pkcs12 -srcstoretype pkcs12 -destkeystore mykeystore

Let's list the contents of this keystore:

keytool -list -storename mykeystore


Add an CA certificate to the default java CA cert keystore

Java keeps its own CA cert repo - which can be frustrating if you don't update java on your server frequently. To add a new CA cert (let's assume you just bought an SSL cert online and your cert CA's certificates were created recently)

1. Find cacerts for your installation - note that many commercial java programs like to install their own java. If you're looking at the default java location, try:

find /usr -name cacerts

1b. If you're trying to find the cacerts for an application you installed, figure out where java is being called from:

ps wwaux | grep java

(you'll likely see a path for java - i.e., /opt/java/bin/java  or something similar)

Now, find the cacerts,  
 find /opt -name cacerts

2. Add the CA cert to your cacerts file:

keytool -import -trustcacerts -alias MyCAsName -file /path/to/ca-certificatec -keystore /path/to/keystore

Friday, September 6, 2013

Cisco ASA to Juniper ScreenOS to Juniper JunOS Reference Cheat Sheet

Here is a basic reference sheet for looking up equivalent commands between a Cisco ASA and a Juniper ScreenOS (or Netscreen) SSG and a Juniper JunOS SRX firewall.
Cisco ASA Juniper ScreenOS (SSG) Juniper JunOS (SRX)
show log get event show log messages
show log messages | last 20 (see the 20 most recent logs)
show ip get interface show interface terse
show failover get nsrp show chassist cluster status
no failover active exec nsrp vsd [vsd] mode backup request chassis cluster failover redundancy-group [group] node
show route get route show route
show connections get session show security flow session
show clock get clock show system uptime
show version (to get uptime) get system show system uptime
show running-config get config show config
show version (to get serial number) get chassis show chassis hardware detail
show access-list get policy show security policies
show crypto isakmp sa get ike cookie show security ike security-associations
show crypto ipsec sa get sa show security ipsec security-associations
clear crypto isakmp sa clear ike cookie clear security ike security-associations
clear crypto ipsec sa clear sa clear security ipsec security-associations
show cpu get perf cpu show chassis routing-engine
show proc cpu-usage
show system processes extensive
show int (for i/o of bytes)  get counter statistics
ssh x.x.x.0 y.y.y.0 inside set admin manager-ip x.x.x.0 y.y.y.0
set ssh enable

show run [cry isakmp|tunnel-group]? get ike gateway
interface Ethernet1
shutdown
set interface ethernet0/0 phy link-down
interface Ethernet1
no shutdown
unset interface ethernet0/0 phy link-down
show failover get nsrp
route outside 1.1.1.0 255.255.255.0 1.1.1.2 set route 1.1.1.0/24 interface bgroup3/0 gateway 1.1.1.2
logging host INSIDE 1.1.1.1
logging trap notification
set syslog config 172.16.200.200 facilities local5
set syslog src-interface ethernet1/0
set syslog enable

ntp server 1.1.1.1 source OUTSIDE set ntp server 1.1.1.1
set ntp server src-interface ethernet3/0
set clock ntp
exec ntp update

capture CAP1 match ip host 1.1.1.1 host 2.2.2.2 clear db
set console dbuf
set ffilter src-ip 1.1.1.1 dst-ip 2.2.2.2
debug flow basic

— OR —

snoop filter ip src-ip 1.1.1.1 dst-ip 2.2.2.2 direction both
clear dbuf
snoop
edit security flow traceoptions
set file TSHOOT
set flag basic-datapath
set packet-filter IN-TO-OUT source-prefix 10.1.1.100/32 destination-prefix 10.2.0.3/32
show capture CAP1 get dbuf stream show log TSHOOT
clear capture CAP1 undebug all
unset ffilter

— OR —

snoop filter delete
deactivate security flow traceoptions
delete security flow traceoptions
Additional reading material regarding Juniper SSG and ScreenOS commands: http://www.juniper.net/techpubs/software/screenos/screenos6.2.0/index.html

Tomcat with apache-mod_jk configuration in Linux


Few years back I was working for a project which have given me a chance to work around Apache and tomcat,there are a few reasons why we’d want to configure Tomcat to run with Apache. For me the most important is security and the fact that I’d like to run my web servers on port 80. In order to run Tomcat on port 80 it has to run as root which is not safe and absolutely not recommended. I am going to show you how to configure your Apache to run and pass connections to Tomcat.
What I normally like to do in such situation is to compile Apache instead of installing from repositories, install and configure Java and Tomcat, and then complile mod_jk to provide Tomcat/Apache connector. Basically all steps will be manual. Perhaps all this can be done using repositories too, but this way I can keep things updated as they updates come out by the developers, not when they become available in repositories.
To simplify the solution and make it a universal tutorial, I am going to refer to each product by its name. Therefore we will be renaming all extracted folders to their generic names. For instance, latest versions of Apache, Tomcat, Java and mod_jk are 2.2.11, 6.0.18, 6u11 and 1.2.27 respectively (Jan 09), but we are going to refer to them as just Apache, Tomcat, Java and mod_jk. I have actually renamed them right after they were extracted before I placed them onto their permanent directories. I am going to use /opt  as my installation directory but you can place them in /opt or any other directory you wish.
Most steps are performed under a non-root account. I will let you know when to use the root account.
Installing Apache:
  1. Log on with your non-root account.
  2. Download Apache tar package from http://www.apache.org and extract. This directory will be called “apache” from this point forward.
  3. Change directory to apache ‘cd apache’.
Prepare the product to compile:
  1. In apache directory:
    ./configure --prefix=/opt/apache
  2. make
  3. Change to root user: su
  4. make install
Once it’s complete run apache:
/opt/apache/bin/apachectl start
test installation by browsing the target address. If it’s running then stop the process:
/opt/apache/bin/apachectl stop
Install Java (I’m using jdk):
  1. Download Java self-extracting package from http://www.java.com. Whatever this file is, we call it java.bin.
  2. Make it executable:
    chmod +x java.bin
  3. Execute:
    ./java.bin
  4. Once you’re done with the installation you will have a directory jre1xxxxx. We will refer to this directory as “java”.
  5. Become root: su (to move directory to /opt)
  6. Move the directory to its final resting place. Mine sits in /opt:
    mv java /opt
Install Tomcat:
  1. Download Tomcat from http://tomcat.apache.org and extract the tar ball. We will call this directory “tomcat” from now on.
  2. Become root: su (to move directory to /opt)
  3. Move this directory to /opt:
    mv tomcat /opt
Now it’s time to configure our paths. Edit your .bash_profile:
STOP: This is local to the user running Tomcat. Each user who’ll be running Tomcat should have a bash_profile in their own home directory.
nano ~/.bash_profile
and add the following lines:
export JRE_HOME=/opt/java
export JAVA_HOME=/opt/java
export CATALINA_HOME=/opt/tomcat
Save and exit, then execute the following command:
source ~/.bash_profile
Now test your Tomcat: as non-root user:
/opt/tomcat/bin/startup.sh
browse to http://localhost:8080. If it’s working then shut it down:
/opt/tomcat/bin/shutdown.sh
Installing connector (mod_jk):
  1. Download mod_jk connector from http://tomcat.apache.org/download-connectors.cgi and extract. We will call this new directory “connector”.
  2. Go to connector/native.
  3. To prepare:
    ./configure --with-apxs=/opt/apache/bin/apxs
  4. make
  5. Become root and:
    make install
  6. Check to see if mod_jk.so is in /opt/apache/modules. If it’s there then you’ve so far been successful.
Create the connector:
Create a file called “connector.conf” in apache/conf directory:
vi /opt/apache/conf/connector.conf
and copy/paste the following lines in that file, save and exit:
workers.tomcat_home=/opt/tomcat
workers.java_home=/opt/java
ps=/
worker.list=myworker
worker.myworker.port=8009
worker.myworker.host=localhost
worker.myworker.type=ajp13
worker.myworker.lbfactor=1
Add the following lines to httpd.conf:
LoadModule jk_module modules/mod_jk.so
JkWorkersFile conf/connector.conf
JkLogFile logs/mod_jk.log
JkLogLevel error
JkLogStampFormat "[%a %b %d %H:%M:%S %Y]"
JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories
JkRequestLogFormat "%w %V %T"
JkMount /*.jsp myworker
JkMount /* myworker
Now, start both Apache and Tomcat. You should be able to view the Tomcat default page in http://localhost AND http://localhost:8080. This means that the connector is working. To increase security disable http access to Tomcat through port 8080.
Note: Apache will automatically start after reboot, but you will have to add Tomcat to your startup script manually. Here is a startup script you can copy to your /etc/init.d and make it executable to start/stop your server. I don’t remember where I got this script from, but just so you know, it’s not mine. Good thing about this script is that even if you run this as root, it will run your Tomcat server as user specified in line 6.
Note: become root to run Apache. If you use the following script for your Tomcat, you may also run it as root. It will switch to specified user once it’s executed. I don’t use the script, however, what I do is to add it to my root crontab and have run it as non-root user. This is the command I use:
su - <username> -c /opt/tomcat/bin/startup.sh
Here is the script to run Tomcat (not mine, I found it in a mailing list):
#!/bin/sh
# Tomcat Startup Script

TOMCAT_OWNER=<user who will run tomcat>; export TOMCAT_OWNER

start() {
        echo -n "Starting Tomcat: "
        su $TOMCAT_OWNER -c $CATALINA_HOME/bin/startup.sh
        sleep 2
}
stop() {
        echo -n "Stopping Tomcat: "
        su $TOMCAT_OWNER -c $CATALINA_HOME/bin/shutdown.sh
}

# See how we were called.
case "$1" in
start)
        start
        ;;
stop)
        stop
        ;;
restart)
        stop
        start
        ;;
*)
        echo $"Usage: tomcat {start|stop|restart}"
        exit
esac

Data dictionaries for FIX Protocol


1) Fiximate
Fiximate is an online FIX reference application provided by fixprotocol.org. I found Fiximate quite easy to use and there description of fix message types, fix tags and other fix protocol related stuff quite helpful. Currently Fiximate is running with version 3.0 and supports FIX.4.0, FIX.4.1, FIX.4.2, FIX.4.3, FIX.4.4, FIX.5.0 and FIX.5.0.SP1 and SP2. You can search here by fix message type, by fix tag number or by tag name. You can even use regular expression while searching. It also has section on various abbreviation used in fix protocol, fix data type and fix fields and component.
2) Fixionary
Fixionary is another fix protocol data dictionary which you can download on your machine. Fixionary is one of the oldest fix data dictionaries available and has good description of all fix protocol information including fix messages, fix tags and fix concepts. There description of fix session and admin message is also very good in order to understand fix protocol concepts.
Transact tool also provide online fix reference guide. Which you can use to refer for getting details on any fix message, fix tag etc. transactool is also a leading provider of commercial fix engines along with NYFIX whose Appia fix engine is one of the most popular commercial fix engines on industry.
Fixopaedia is a fix data dictionary from btobits another commercial fix engine vendor. Fixopadedia is available online and you can access it with the link provided here. This fix data dictionary supports almost all version of fix protocol include fix.4.2, fix.44, FIX 5.0 SP1 AND fix5.0sp. Fixopaedia also has nice documentation around component block, message names, message types, field name and field types. Btobits also provides some next generation fix protocol tools which helps in development and fix protocol certification.
5) Onixs fix data dictionary
This is another online fix data dictionary, my personal favorite because of nice site layout and quality information related to fix messages and tags. You can browse FIX Dictionary by FIX Protocol version, by MsgType, and by Tag number and tag Name. This fix data dictionary not only serves as online fix reference but also a good fix protocol tutorial for beginners and advanced developers.

Linux networking basics for FIX Connections

Knowledge of networking basics are extremely important if you are dealing with FIX connection , FIX stands for Financial Information Exchange protocol and its the most popular protocol for electronic trading and supported by many brokers , exchanges etc.

  I would like discuss few important network related information which is quite useful for FIX connection establishment.


VPN (Virtual Private Network)

------------------------------
VPN uses a technique known as tunneling to transfer data securely on the Internet to a remote server on your workplace network. Using a VPN helps you save money by using the public Internet instead of using costlier leased lines. VPN provides security at network layer unlike HTTPS which provide security at application layer.
most of clients and broker used VPN to connect there FIX Engines and allow trading.

before setup of VPN some contracts needs to be sorted out by both party
e.g.
1) Encryption key
2) Protocol used
3) VPN Connector IP

once your VPN connection got setup you can connect to your brokers trading system on there respective IP/Port.

DMZ (Demilitarized Zone)
--------------------------------
Its host on any network which is used to connect Corporate network to public network e.g. external exchange connection.
Firewall rules are relaxed for this zone , also if security of DMZ is compromised then also intruder won’t get access to corporate network.

Normally client connection or exchange/broker connection originates and terminates in DMZ.

.

How to use Regular Expression

One of the most powerful tools available to sysadmins/programers  are Regular Expressions, if you are good with regular expressions you can solve many day to day problem extremely quickly e.g. find all the lines with ERROR in log file , find count of any particular ID in log file , find exceptions etc.

regular expression is generic concept which has implemented on many different language and many different tools including Java, here is some of the tools and languages which use regular expression.

  • The vi editor which comes standard with the Unix/Linux operating system.
  • Any decent programmer's editor e.g. EditPlus,Notepad ++
  • The grep command found standard on many operating systems including Unix/Linux
  • Sed command can be found in Unix/Linux
  • The Perl programming language.  
  • The PHP programming language. 

Regular Even if you feel regular expression is complex and hard to learn I would suggest to familiarize with atleast basic set of regex and try to use it as much as possible and later you will only want to learn more and more to do the stuff quickly.

Here are some of the basic regex and there examples:

1) you want to find ERROR in log file in linux ?    
grep ERROR logfile

2) If you want to find lines starts with ERROR ?
grep ^ERROR logfile     (^ is used to find for startswith)

3) If you want to find lines ends with ERROR ?
grep ERROR$ logfile     ($ is used to find for endswith)

4) If you want to find empty lines in log file ?
grep ^$ logfile

5) If  you want to match upper or lower case ERROR ?
grep [Ee]RROR logfile               ([] is used to include letters]

6)If you want find all lines that contains Error or Exception ?
egrep ERROR|Exception logfile   ( | is used for OR condition)

7) Match the letter E when it appears at least 3 times in a row but possibly 4 or more times in a row: E {3,} 

8) Match the letter E when it appears 3 times in a row or 6 times in a row or anything in between. E{3,6}

9)Match E when it appears 1 or more times in a row.
E+ 

10) (E+ and E{1,} mean exactly the same thing!)