sorting by date in unix

sort -t'/' -k3n -k2M -k1n

Passing Environment Variables in Sudo

As a normal user, we have to use privilege-gaining tools such as sudo to run programs as the root user when required. With the super user rights in hand, are we still working in the same environments set by the normal user ?

The reason why I ask this question is that I’m losing the environment variables set in ~/.bashrc when running a bash script with sudo. For instance, the variable JAVA_HOME is set and exported in ~/.bashrc.

export JAVA_HOME=/usr/lib/jvm/java-6-sun

A bash script called example.sh will use this variable to find java home directory.

#!/bin/bash

echo $JAVA_HOME

When example.sh is invoked by the current user by typing

./example.sh

the output is exactly the expected one:

/usr/lib/jvm/java-6-sun

However, when trying with

sudo ./example.sh

We get nothing. The variable is lost or in a other word, not inherited.

Solution:

Use the following sudo option

sudo -E ./example.sh

You could also use an interactive sub shell, but thats not a recommended option. I will provide that option any ways..

invoke an interactive sub shell

#!/bin/bash -i

echo $JAVA_HOME

Using Xming as your Xserver / Xclient

Installing PuTTY and Xming

With PuTTY and Xming you can run text and graphical programs on our servers from your own PC.

Software you need to download and install:

1. Xming-fonts (local copy)
2. Xming (local copy)
3. PuTTY (local copy)

[Local copyies where last updated at 05-04-2009]

Install the software above in the listed order with their default settings.

Configure PuTTY

You need to enable X11 forwarding to show graphical programs:

1. Start PuTTY.
2. In the left menu go to: Connection > SSH > X11.
3. Check "Enable X11 forwarding".
4. In the left menu go to: Window > Translation.
5. Select UTF-8 in the drop down box.
6. Go back to "Session" in the top of the left menu.
7. Click "Default Settings" and Save.

Configure Xming

If you don't what to remember to start Xming evertime you what to use it, you can set it to start when Windows startup. To do this, copy the Xming shoutcut from the start menu to the startup folder in the start menu.

Connecting to a server

1. Find an appropiate server from the server list.
2. Make sure Xming is running.
3. Launch PuTTY.
4. Type the server name in the host name field.
5. Optimal: Save your session, so you only have the double click on the name next time.
6. Click Open.

If everything is working, a window with a black backgound will open, prompting you for a user name and password. Try to start a graphical program like Emacs or Matlab.

Troubleshooting

Problem: When I try to start a graphical program, I get "Connection lost to X server".
Solution: Check that Xming is started.

Problem: When I try to start a graphical program, I get "connection refused by server".
Solution: Make sure you have a token by using the "klog" command.

Problem: Programs start up in text mode.
Solution: Check that "X11 forwarding" is enable in PuTTY. If you use a stored session, load it, enable "X11 forwarding", and save it.

Problem: Letters are displayed as boxes in graphical programs.
Solution: Install Xming-fonts and restart Xming.

Problem: Graphical programs are very slow.
Solution: Enabling "SSH compression" might help if you use a low bandwidth connection. The option is under Connection > SSH in the left menu of PuTTY.

Problem: In Matlab backspace and return does not work.
Solution: Turn off numlock.

Tips to reduce WebLogic Startup Time

This is going to be a living document on how to reduce weblogic startup times.

1) Apply -D_Offline_FileDataArchive=true

The WebLogic Diagnostics Framework creates the diagnostic store (DAT) file which is used for different purposes. The initial size of the store file is about 1M.

Here are the details of the parameter suggested: -D_Offline_FileDataArchive=true

The above parameter is a (undocumented and unsupported) system property that disables WLDF indexing on the log file.

Engineering has delivered an undocumented system property to disable the file archive indexing task (-D_Offline_FileDataArchive=true),
This property will disable incremental as well as full indexing of WLDF archiving, if property is not properly applied, you can see a File Indexer timer kicks in for every 30 seconds.

The parameter is discussed in the bug# 8101514, where customer has encountered High CPU usage because of this indexing and it has resolved couple of Production issues as well.

The indexing takes place when ever we create a WLDF module.

URL for WLDF: http://download-llnw.oracle.com/docs/cd/E13222_01/wls/docs100/wldf_configuring/index.html

2) WLW page-flow event reporter may be creating several entries in the diagnostics store

apply -Dcom.bea.wlw.netui.disableInstrumentation=true

3) Disable JDBC profiling. (Under JDBC/datasource_name/diagnostics uncheck all the options there)

If JDBC profiling is turned on, it inserts profiling data into the diagnostic store, which can cause it to grow.

URL for JDBC profiling: http://download-llnw.oracle.com/docs/cd/E13222_01/wls/docs100/ConsoleHelp/pagehelp/JDBCjdbcdatasourcesjdbcdatasourceconfigdiagnosticstitle.html

4) Disable Harvesting (change the value for Profile Harvest Frequency Seconds to zero)

5) Try setting the "synchronous-write-policy" parameter in the server configuration to "Cache-Flush". This bypasses the "direct I/O" code in the file store.

Servers--->Configuration--->Services--->Default Store you can set the synchronous-write-policy to Cache-Flush

Synchronous Write Policy:

The disk write policy that determines how the file store writes data to disk.

This policy also affects the JMS file store's performance, scalability, and reliability. The valid policy options are:

* Direct-Write

File store writes are written directly to disk. This policy is supported on Solaris, HP-UX, and Windows. On Windows systems, this option generally performs faster than the Cache-Flush option writes.

* Disabled

Transactions are complete as soon as their writes are cached in memory, instead of waiting for the writes to successfully reach the disk. This policy is the fastest, but the least reliable (that is, transactionally safe). It can be more than 100 times faster than the other policies, but power outages or operating system failures can cause lost and/or duplicate messages.

* Cache-Flush

Transactions cannot complete until all of their writes have been flushed down to disk. This policy is reliable and scales well as the number of simultaneous users increases.

6) If you are using jrockit under GNU/Linux, you might be hitting a bug. Inside jrockit installation directory, jre/lib/security, edit java.security and locate the line that defines the random source:

securerandom.source=file:/dev/urandom

Comment it out, and replace it by:

securerandom.source=file:/dev/./urandom

7) You can also break out the JMS servers from the instance and allocate them into a seperate managed server - WLS would have to read the JMS file store on every start
Or compact the JMS file store regularly to keep its size small

How does WebLogic Serve Requests

1. A client contacts the ListenThread, the entry point into WebLogic Server, which accepts the connection. It then registers the socket with a WLS component known as the SocketMuxer for further processing.

2. The SocketMuxer is responsible for reading and dispatching all client requests to the proper WLS container. It then adds this socket to an internal data structure for processing and makes a request of an ExecuteThreadManager to create a new SocketReaderRequest. This request is then dispatched by the manager to an ExecuteThread

3. As a result, the ExecuteThread becomes a SocketReader thread - it will continually run the SocketMuxer’s processSockets and checks the muxer’s queue to determine if there is work to be done. If an entry exists, it pulls it off the queue and processes it.

4. The SocketReader thread reads the client request and determines the protocol type of this client request, to create a new protocol specific MuxableSocket.

5. The MuxableSocketDiscrminator stores a new MuxableSocket with the implementation matching the protocol of the client. It also returns true to the SocketReader to notify it that the message is complete and it can be dispatched.

6. MuxableSocketDiscriminator re-registers the protocol specific version of the MuxableSocket that was created earlier. The net result is that “Step 2” is repeated, and a new protocol specific MuxableSocket is placed in the SocketMuxer’s queue for processing.

7. A socket reader will get the new protocol specific MuxableSocket off the queue and read it. It will then checks to see if the message is complete, based on the protocol. If it is, it will invoke the protocol specific MuxableSocketDiscriminator

8. Before the work requested by the client can be performed, there may be many iterations of “step 7”. This is determined by the protocol – for example, t3 will read a portion of the message, dispatch it so it can act upon the portion of the protocol read thus far.

9. The subsystem will create an ExecuteRequest and send it to an ExecuteThreadManager for processing. The request is dispatched to an ExecuteThread, and the result is returned to the client.


From a high level overview’s perspective, the SocketMuxer can be explained as follows. Each and every socket connection that comes into WebLogic Server is “registered” with the SocketMuxer - which then maintains a list of these connections, each represented by a form of the MuxableSocket interface. It then becomes the responsibility of the SocketMuxer to read and dispatch each client request to the appropriate subsystem. This is a fairly elaborate process, which is illustrated by steps 2 through 8 above.

There are only a few key things to know about the SocketMuxer:

First, it has a data structure in which it stores a socket entry for each client connected to WebLogic Server.
Second, a “socket reader” is the main component of the SocketMuxer - which really is just an execute thread that is running the SocketMuxer’s processSockets() method.

Third, the SocketMuxer does most of its work through the only interface it knows how to operate on – the MuxableSocket interface.

Socket Reader:

A SocketReaderRequest is merely an implementation of an ExecuteRequest, which is sent to the ExecuteThreadManger by the invocation of the registerSocket(). When the ExecuteThread invokes the execute() method of the SocketReaderRequest, the SocketMuxer’s processSockets() method is invoked.
So, a socket reader thread is simply a normal execute thread which runs the main processing method of the SocketMuxer, processSockets().


The acceptBacklog parameter of Weblogic server is passed to ServerSocket. The value of acceptBacklog means "The maximum queue length for incoming connection indications (a request to connect) is set to the backlog parameter. If a connection indication arrives when the queue is full, the connection is refused. "

Thus if too many connects come on the server at the same time, the server would queue this connects and process them one at a time. The value does not mean that only that many clients can connect to the server.

It does not limit the number of connections made. It limits the number of potential connections that can lie in the backlog queue. So for e.g.: AcceptBacklog is 2. If hundreds of connections were made to the server and the server has one thread to accept new connections.

This thread accepts the new connection and dispatches it to a new thread and then goes back to listening to new connections. Sample code is

while (true) {

Socket sock = serversocket.accept(); // Line 1
new MyThread(sock).run(); // Line 2

}

Here the thread accepts a new connection at line1. Dispatches to new thread in line 2. Evaluates the while expression and goes back to line 1. So in between the time it takes for it to get back to line 1(say T1) many new connections requests are made by the clients. These new connections lie on the accept backlog queue and this queue length is controlled by the accept backlog parameter.

If the queue length is 2, and between this time T1 several hundred connections are made to the server only 2 would get accepted and rest of them rejected. For rejection there must be too many simultaneous requests to the server, if it’s not simultaneous then the chances of queue getting full is less.