Monday, September 11, 2017

Bash script to repeat a task until you tell it to stop

For example, when I am running some task that outputs to the database but takes time to finish, rather than running the SQL myself over and over, I will script it and run the script to repeat. Then I hit ENTER whenever I am ready to find out when it has finished. This becomes more effective when I script the whole task: start the asynchronous job that takes awhile, then repeat-run the monitoring script.

#!/bin/bash
weContinue=true
while "${weContinue}" == true ; do
   "$@"
   echo "[$(date)] Press ENTER to continue; press any other key (then ENTER) to stop."
   read response
   if [ -n "${response}" ] ; then
      weContinue=false
   fi
done

Example running it.

Mon Sep 11 - 11:11 AM > repeat echo "Do something important"
Do something important
[Mon, Sep 11, 2017 11:19:54 AM] Press ENTER to continue; press any other key (then ENTER) to stop.

Do something important
[Mon, Sep 11, 2017 11:19:58 AM] Press ENTER to continue; press any other key (then ENTER) to stop.

Do something important
[Mon, Sep 11, 2017 11:20:01 AM] Press ENTER to continue; press any other key (then ENTER) to stop.
stop thanks

Sunday, July 30, 2017

Scripts and .screenrc to make GNU Screen splits easier

Update Thursday, 3rd of August 2017, 11:04:30 AM: much better to use bind over stuff to run a script within the .screenrc file.

I enjoy GNU screen via Cygwin on Windows a lot, but some of the commands get a bit fiddly. Creating splits is like that: you create a split, then move to it, then assign a window to it.

My first set of shortcuts were to create scripts to do those jobs.

To make a vertical split:

To make a horizontal script:

Lastly, key bindings that go in my .screenrc file to run those scripts.

# ------------------------------
# SPLIT HORIZONTALLY OR VERTICALLY.
# ------------------------------
# Uses split scripts.
# Control+a, V for vertical; control+a, H for horizontal.
# bind V stuff 'screenSplitVertical'\012''
# bind H stuff 'screenSplitHorizontal'\012''
bind V exec $HOME/bin/screenSplitVertical.sh
bind H exec $HOME/bin/screenSplitHorizontal.sh

Explanation for version that uses exec:

  1. bind V says the rest of this line will be executed upon "control+a, shift+v"
  2. exec means run the rest of the line as a command.
  3. $HOME/bin/screenSplitVertical.sh is the full path to the command (script) I want to run.

Explanation for version that uses stuff:

  1. bind V says the rest of this line will be executed upon "control+a, shift+v"
  2. stuff will "write stuff to the command line" or "stuff text onto the command line".
  3. 'screenSplitVertical'\012'' will write out the command screenSplitVertical.sh and then '\012' which outputs a newline (ENTER key) to actually cause the script to run.

In general, exec is more suitable than stuff in this situation because stuff will leave the command in my command history.

More screen magic? Check out joaopizani's .screenrc config for some great ideas about resizing splits and moving between them.

Sunday, November 08, 2015

Maven project with Spring, Log4j2 and properties

This a reference I use to quickly come up with a standalone app (no server) that uses Spring Boot, Log4j2 and a properties file.

The pom.xml file.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
   <modelVersion>4.0.0</modelVersion>
   <groupId>org.testproject</groupId>
   <artifactId>TestProject</artifactId>
   <packaging>jar</packaging>
   <version>1.0-SNAPSHOT</version>
   <name>TestProject</name>
   <url>http://maven.apache.org</url>

   <properties>
      <maven.compiler.source>1.8</maven.compiler.source>
      <maven.compiler.target>1.8</maven.compiler.target>
      <jodatime.version>2.5</jodatime.version>
      <junit.version>4.12</junit.version>
      <log4j.version>2.4.1</log4j.version>
      <spring.boot.version>1.2.5.RELEASE</spring.boot.version>
   </properties>

   <dependencies>
      <dependency>
         <groupId>junit</groupId>
         <artifactId>junit</artifactId>
         <version>${junit.version}</version>
         <scope>test</scope>
      </dependency>
      <dependency>
         <groupId>org.apache.logging.log4j</groupId>
         <artifactId>log4j-core</artifactId>
         <version>${log4j.version}</version>
      </dependency>
      <dependency>
         <groupId>org.springframework.boot</groupId>
         <artifactId>spring-boot</artifactId>
         <version>${spring.boot.version}</version>
      </dependency>
      <dependency>
         <groupId>org.springframework.boot</groupId>
         <artifactId>spring-boot-autoconfigure</artifactId>
         <version>${spring.boot.version}</version>
      </dependency>
   </dependencies>

</project>

The configuration class that launches the app.

package org.testproject;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;

/**
 * Configuration and runner for the test app.
 */
@Configuration
@ComponentScan(basePackages = { "org.testproject.*" })
@EnableAutoConfiguration
public final class TestApp {

   /**
    * Launch app.
    *
    * @param args
    *           not used
    */
   public static void main(final String[] args) {
      SpringApplication.run(Runner.class, args);
   }

}

The runner - the class that does the actual work.

package org.testproject;

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.CommandLineRunner;
import org.springframework.context.annotation.PropertySource;
import org.springframework.context.annotation.PropertySources;
import org.springframework.core.env.Environment;
import org.springframework.stereotype.Component;

/**
 * Read in a message file from Touch Copy and transform the contents to be diary
 * ready.
 *
 * @author RobertMarkBram
 */
@Component
@PropertySources(value = {@PropertySource("classpath:/application.properties")})
public final class Runner implements CommandLineRunner {

   /** Logger for this class. */
   static final Logger LOG = LogManager.getLogger(Runner.class
         .getName());

   /** Use to access properties. */
   @Autowired
   private Environment env;

   /* (non-Javadoc)
    * @see org.springframework.boot.CommandLineRunner#run(java.lang.String[])
    */
   @Override
   public void run(final String... args) throws Exception {
      String testproperty = env.getProperty("testProperty");
      LOG.info(testproperty);
   }

}

The application property file.

testProperty=Hello World

The log4j2 configuration - note that number 2 is there.. this not old log4j anymore.

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="INFO">
   <Appenders>
      <Console name="CONSOLE" target="SYSTEM_OUT">
         <PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n" />
      </Console>
   </Appenders>
   <Loggers>
      <logger name="org.testproject" level="DEBUG" />
      <Root level="ERROR">
         <AppenderRef ref="CONSOLE"/>
      </Root>
   </Loggers>
</Configuration>

Tuesday, September 15, 2015

Date time stamps in bash with ordinal indicator

Update (Wednesday, 4th of November 2015, 12:15:35 AM): added link to more complicated script.

Date time stamps are easy in bash:

> echo $(date +"%Y%m%d_%H%M%S")
20150915_214916

But I want more friendly date time stamps, like Tuesday, 15th of September 2015, 09:54:42 PM.

I found an algorithm that Greg Mattes wrote in this StackOverflow answer for Java. Here it is in bash:

#!/bin/bash
n=$(date +"%d")

if [ $n -ge 11 -a $n -le 13 ] ; then
  echo "th"
else
 case $(( $n%10 )) in
 1)
   echo st
   ;;
 2)
   echo nd
   ;;
 3)
   echo rd
   ;;
 *)
   echo th
   ;;
 esac
fi

So now I can do:

> echo $(date +"%A, %d`dateOrdinal.sh` of %B %Y, %I:%M:%S %p")
Tuesday, 15th of September 2015, 09:57:53 PM

Here is a more complicated version of this script that will accept a date, a date format and output formatted date where you can use %O as a placeholder for the ordinal in the format: http://pastebin.com/xZ1afqqC.

Thursday, June 18, 2015

JQuery UI tooltip with a close link for mobile

Here is the JQuery code I use to to set up JQuery UI tool tips with a close link that only appears on mobile platforms. This is useful because on a desktop, the tool tip will close when you mouseout from the target element, but on a touch platform it sticks around.

// Set up tool tips for images and anchors.
$( document ).tooltip({
   items: "a[title], img[alt], .toolTip[title], :not(.noToolTip)",
  track: true,
  position: { my: "left+15 center", at: "right center" },
   content: function() {
      var element = $( this );
      var closer = closerLink = '';
      if (isMobile()) {
         closer = ' <br><div onClick="$(this).parent().parent().remove();" style="color: blue; text-decoration: underline; text-align: right;">Close</div>';
         closerLink = ' <br>Tap link again to open it.<br><div onClick="$(this).parent().parent().remove();" style="color: blue; text-decoration: underline; text-align: right;">Close</div>';
      }
      // noToolTip means NO TOOL TIP.
      if ( element.is( ".noToolTip" ) ) {
         return null;
      }
      // Anchor - use title.
      if ( element.is( "a[title]" ) ) {
         return element.attr( "title" ) + closerLink;
      }
      // Image - use alt.
      if ( element.is( "img[alt]" ) ) {
         return element.attr( "alt" ) + closer;
      }
      // Any element with toolTip class - use title.
      if ( element.is( ".toolTip[title]" ) ) {
         return element.attr( "title" ) + closer;
      }
   }
});

function isMobile() {
   return (/iPhone|iPod|iPad|Android|BlackBerry/).test(navigator.userAgent);
}

I am targeting three types of things here:

  • Anchor tags (a) with a title attribute.
  • Image tags (img) with a title attribute.
  • Any element with class toolTip.
  • And specifically exclude any element with class noToolTip.

You can see this in action on my newly refreshed site: Chihuahua Rescue Victoria.

Wednesday, June 17, 2015

Run main class in test dir from maven with Java agent and Log4J configuration

It took a bit of help from Michal (on this StackOverflow post: Running main method from test class via maven) before I got this right. But here it is:

<profile>
   <id>run-importer</id>
   <properties>
      <loadTimeWeaverArg>-javaagent:"${settings.localRepository}/org/springframework/spring-agent/2.5.6/spring-agent-2.5.6.jar"</loadTimeWeaverArg>
      <log4JConfigArg>-Dlog4j.configuration=file:${project.build.directory}/path/to/log4j.properties</log4JConfigArg>
      <mainClassArg>com.myorg.MyClass</mainClassArg>
      <arg1>foo</arg1>
   </properties>
   <build>
      <plugins>
         <plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>exec-maven-plugin</artifactId>
            <version>1.2.1</version>
            <goals>
               <goal>exec</goal>
            </goals>
            <configuration>
               <executable>java</executable>
               <classpathScope>test</classpathScope>
               <arguments>
                  <argument>${log4JConfigArg}</argument>
                  <argument>${loadTimeWeaverArg}</argument>
                  <argument>-classpath</argument>
                  <classpath />
                  <argument>${mainClassArg}</argument>
                  <argument>${arg1}</argument>
               </arguments>
            </configuration>
         </plugin>
      </plugins>
   </build>
</profile>

And I run it with the command line: mvn -e exec:exec -Prun-importer

The advantages of this approach:

  • The whole purpose of this profile is to run "special code" that should never be deployed but needs to make use of code in src and test src.
  • It leaves room in case this ever needs to be duplicated, so there is no "competition" regarding what gets run with mvn -e exec:exec.
  • I can specify java agent, log4j and lots of other config using variables that already exist in the pom.
  • I can override any of these arguments on the command line with -Darg1="bar"

Sunday, April 26, 2015

Generating getters setters toString hashCode and equals

Generating code is a good way to remove so much boilerplate in Java that you would otherwise write by hand. Getters and setters is a big one. Eclipse can generate those, but Eclipse generation clashes with Checkstule rules that I use. The toString(), equals() and hashCode() methods are important to get right too. Previously I set up Eclipse templates but I still had to fill in the details by hand. Most of the time it's better to start with a default implementation for each that uses every non-static field in the class.

This post demonstrates some code I have written to automate the generation of these things.

Checkstyle vs Getters and Setters

I use Checkstyle wherever I can. It encourages me to write code with a consistent style and, like FindBugs, helps me to avoid common errors. Here are some Checkstyle rules that I use:

  1. Ensure that I have a jadvadoc comment on fields and methods. This includes checking that I have a @return tag for non-void methods and a @param tag for each parameter. Javadoc comments let me impart important context that cannot always be achieved by self documentating code.
  2. Final parameters on methods. Avoid accidentally re-assigning a parameter during method execution by making sure that all parameters are delcared as final. If I need a primitive (or wrapper or other immutable object like String) whose value will change, it is clearer to define a new local variable and assign it an initial value from the corresponding final parameter if needed.
  3. Avoid hidden fields. Don't have a parameter with the same name as an instance variable, because it is too easy to accidentally modify the wrong one. Using this on the instance variable will avoid these errors, but it is easy to forget.

In my opinion, these are very good rules to live by when coding and as I write new methods, I find myself automatically implementing these rules: they have become muscle memory for me.

However these rules have a cost when dealing with getters and setters. Let's say I create a new class, add ten fields, and in Eclipse select Source > Generate getters and setters > select all > OK and bam, I have ten corresponding accessor and mutator methods for my ten fields. Now I would have to edit twenty methods to correct the comments and edit ten methods to make parameters final and give them a different name. Here is an example of the getters and setters generated by Eclipse.

private String name;


private int id;


/**
 * @return the name
 */
public final String getName() {
   return name;
}


/**
 * @param name the name to set
 */
public final void setName(String name) {
   this.name = name;
}


/**
 * @return the id
 */
public final int getId() {
   return id;
}


/**
 * @param id the id to set
 */
public final void setId(int id) {
   this.id = id;
}

Eclipse generation cannot, as yet, make parameters final or give them different names, and the comments are very basic and only use the parameter name to make them different. It is less likely that following the Checkstyle rules will create as much value on getters and setters as on other methods. Getters and setters are very simple methods, usually one-liners. Because they are so simple that they can be auto-generated, we don't generally modify them after creation - so there is less need to give them the protection of checkstyle rules like final parameters and avoiding hidden fields. Thus, the cost of editing so many methods is rarely worth the value it gives.

Having said all that, there a couple of reasons why I still want to follow these rules, even on getters and setters if I can cut the cost significantly.

  • Comments are still valuable for fields/getters/setters that cannot be truly un-ambiguously named, which happens often enough no matter how much thought you can put into it. For the simple ones, simple comments are fine.
  • If my Eclipse Problems or Markers view is filled up with dozens of Checkstyle warnings from getters and setters, it is harder for me to spot the warnings I really should take care of. So I would prefer to keep my Problem/Markers view clean and automatically implement the Checkstyle rules if possible.

Generate Boilerplate

So here is how I have attempted to fix this issue and make it as painless as possible to generate getters and setters and implement the Checkstyle rules I want. At the same time, I will generate decent default toString(), equals() and hashCode() methods to copy and paste too.

As a pre-requisite, I have checked out my utils project from GitHub into Eclipse: https://github.com/robertmarkbram/RobUtils so that I can use three classes in particular:

  • OutputFieldList.java which has some utility functions that use reflection to inspect a class and do things I sometimes find useful such as
    • output a list of fields or getters and setters that are present in a class.
    • output sample equals(), hashCode() and toString() impelementations using the commons lang builders.
    • outputs sample getters and setters from fields defined in a class.
  • FieldComment.java which is an annotation that stores a field's comment that OutputFieldList will read and output as javadoc comments for getter and setter fields. It's based on a solution Abhi wrote about in this StackOverflow answer to the question: How to get a JavaDoc of a method at run time?
  • FieldCommentSampleClass.java which is just a place I copy the field declarations into so that I can use the FieldComments annotation in the same project as the other things in RobUtils.

I begin by writing out the fields in the actual class they belong to. I write out comments for each.

public class Foo {
   /** User friendly label for display. */
   private String name;

   /** Unique ID within datastore. */
   private int id;

}

I then copy these comments into FieldCommentSampleClass.java, which is just a scratch pad that has the advantage of being in the same project as the @FieldComment annotation. Within FieldCommentSampleClass.java, I do a couple of find and replace operations to change the javadoc comments to @FieldComment annotations.

  1. Replace /**  with @FieldComment(comment=".
  2. Replace */ with ").

So that now have fields with the @FieldComment annotation:

public class FieldCommentSampleClass {
   @FieldComment(comment="User friendly label for display. ")
   private String name;

   @FieldComment(comment="Unique ID within datastore. ")
   private int id;
}

I then run the main method in OutputFieldList.java with these two calls. This will generate getters and setters, plus decent toString(), hashCode() and equals() methods for me.

public static void main(final String[] args) throws Exception {
   generateGetAndSetMethods(FieldCommentSampleClass.class, //
      OutputFields.NO_FIELDS, //
      OutputGetters.OUTPUT_GETTERS, //
      OutputSetters.OUTPUT_SETTERS);
   outputToStringHashAndEquals(Foo.class);
}

I now have the following output that I copy and paste into my original class.

/** @return user friendly label for display.  */
public String getName() {
   return name;
}

/** @return unique ID within datastore.  */
public int getId() {
   return id;
}


/** @param theName user friendly label for display.  */
public void setName(final String theName) {
   this.name = theName;
}


/** @param theId unique ID within datastore.  */
public void setId(final int theId) {
   this.id = theId;
}


@Override
public boolean equals(final Object obj) {
   if (obj == this) {
      return true; // test for reference equality
   }
   if (obj == null) {
      return false; // test for null
   }
   if (obj instanceof FieldCommentSampleClass) {
      final FieldCommentSampleClass other = (FieldCommentSampleClass) obj;
      // @formatter:off
      return new org.apache.commons.lang.builder.EqualsBuilder()
            .append(name, other.name)
            .append(id, other.id)
            .isEquals();
      // @formatter:on
   } else {
      return false;
   }
}


@Override
public int hashCode() {
   // @formatter:off
   return new org.apache.commons.lang.builder.HashCodeBuilder()
         .append(name)
         .append(id)
         .toHashCode();
   // @formatter:on
}


@Override
public String toString() {
   org.apache.commons.lang.builder.ToStringBuilder.setDefaultStyle(org.apache.commons.lang.builder.ToStringStyle.SHORT_PREFIX_STYLE);
   // @formatter:off
   return new org.apache.commons.lang.builder.ToStringBuilder(this)
      .append("name", name)
      .append("id", id)
      .toString();
   // @formatter:on
}

Final notes

A few notes to keep in mind.

  • OutputFieldList needs JDK 8 because it uses the Stream API with lambdas.
  • The toString(), equals() and hashCode() methods generated use the fully qualified class names for the apache commons lang builders. I them use Source > Add Import (or control+shift+m) to replace the fully qualified class names with just the class names.
  • If the code I am generating boiler plate for relies on custom code (e.g. field types are not standard java types), I may have to modify my RobUtils project so that it can reference those types, which is easy: Project > Properties > Java Build Path > Projects tab > Add and select the projects I need to reference.
  • I am pragmatic about following these Checkstyle rules.
    • When editing code written by someone else that has Checkstyle warnings, should I fix them? Perhaps. If there are just a few and I have time, sure. If there are many then only fix the ones in the actual lines of code I need to edit.
    • I don't force my own practices on other developers in a team. If the team has a policy already in place around the use of tools like Checkstyle or Findbugs, that takes precedence. Otherwise I practice what makes sense to me. If other developers are interested, explain how I use the tooling to help me write good code, but don't proselytise.
    • As I mentioned above, there is a cost to taking on processes like Checkstyle rules. If the cost is too high (i.e. there is little benefit to be gained), or the rules doesn't make sense to you, don't do it. Checkstyle warnings can be suppressed for a class, method or code block and rules can be removed entirely from a project specific configuration if the rule is too burdensome.