Wednesday, December 17, 2008

Calling a Site Entry from a form

Updates
16th November, 2009: or use action=post.

In Content Server Explorer, you can create a JSP that is addressable through a Site Entry. For example, create a Content Server Element named my/site/FormResponse and a Site Entry whose RootElement is my/site/FormResponse.jsp - and you can reach that element with a URL such as: http://SERVER/servlet/ContentServer?pagename=my/site/FormResponse. Content Server is the controller (/servlet/ContentServer) that invokes other elements based on the GET arguments sent to it :?pagename=my/site/FormResponse for example.

That's good, but you cannot use that URL directly in a form's action attribute, such as below.

<FORM ACTION="http://SERVER/servlet/ContentServer?pagename=my/site/FormResponse">
<INPUT TYPE="Submit" VALUE="Submit"/>
</FORM>

That's because in a form's action attribute, any GET style arguments at the end of the URL are ignored. So the URL invoked from the FORM above would be http://SERVER/servlet/ContentServer. The fix is simple enough. Put the pagename argument in a hidden field, such as below.

<FORM ACTION="http://SERVER/servlet/ContentServer">
<INPUT TYPE="Submit" VALUE="Submit"/>
<INPUT TYPE="Hidden" NAME="pagename" VALUE="my/site/FormResponse"/>
</FORM>

Or (much simpler!) just explicitly specify action as POST.

<FORM ACTION="http://SERVER/servlet/ContentServer?pagename=my/site/FormResponse" action="POST">
<INPUT TYPE="Submit" VALUE="Submit"/>
</FORM>

Saturday, December 13, 2008

Cygwin Bash script: search through files and let user open results

I frequently use this Bash script in Cgywin to easily search for a file by name (using regex), see a list of results and choose which of those results I want to open up in my favourite text editor, like UltraEdit. The script in this post my own wrapper for grep and find that allows me to search through the contents of files, show the results of the search and call the previous script to show me a list of matching files, asking me which ones I want to open.

Purpose. Search through files, show user results and allow user to open any of the files that contained a match.

Input. File name search criteria, file contents search criteria, selection of what files to output.

Output. Lines of results, sorted by file, menu of result files to open.

Dependencies. Relies on this Bash script being present in the path - refers to it as u - because when I first wrote it, I would only use it to open files in UltraEdit!

Example 1.

  1. Search through all *.java files for the regular expression setco.*nager.
  2. Display results sorted by file.
  3. Allow user to see a list of result files and select which ones, if any, to open.
-bash3.2 - rbram@MCLL - Javascript TOC - /cygdrive/c/temp/DocPublisher
Sat Dec 13 - 08:13 PM > search --type j --term "setco.*nager"
Search for pattern "setco.*nager" in dir /cygdrive/c/temp/DocPublisher through file pattern "*.java"

====
./src/java/au/com/publisher/doc/PublisherServiceImpl.java
846:    public void setCollectionManager(

====
./test/java/other/au/com/publisher/admin/MockData.java
1005:           publisherServiceImpl.setCollectionManager(mockCollectionManager);

Do you want to view any of the matching files?
============
File  0: ./src/java/au/com/publisher/doc/PublisherServiceImpl.java
File  1: ./test/java/other/au/com/publisher/admin/MockData.java
Specify files to open. [A]ll, [N]one or [x y z] space separated indexes.

Code

#!/bin/bash

#-------------------------------------------------------------------------------
#  Search Command
#-------------------------------------------------------------------------------
# Search through files in any location and open any of them in an editor.
#
# Author: Robert Mark Bram
# v 1.2 - 1920/11/2007 1:37:05 PM. Included the ability to follow (or not) symbolic
#   links by hooking into the find command's capability for this. By default, do
#   not follow symbolic links.
# v 1.3 - 29/02/2008 4:03:53 PM. Don't look in files matching:
#   grep -v '[.]svn-base$\|zzbuild'

#-------------------------------------------------------------------------------
#  Dependencies
#-------------------------------------------------------------------------------
# - Relies on the 'u' command.

#-------------------------------------------------------------------------------
#  Variables for this script.
#-------------------------------------------------------------------------------
# Variables for this script.
# Pretty up the output to make it more readable. Yes or no.
pretty=yes
# Make the search case sensitive (-i) or not (no value).
case=-i
# Output results to console. Yes or no.
console=yes
# Pattern of files in which we shall search.
filePattern=
# Output names of files only.
fileNamesOnly=
# Number of lines around the search result to display.
numberOfLines=0
# Directory within which to search.
searchDir=.
# Was --type given to specify a file type? This var will have that value.
filePatternType=
# Should filePattern be treated as a regular expression as per find: -iregex pattern?
filePatternRegularExpression=N
# Follow symbolic links? -P (no) or -L (yes) - as per the find command.
symbolic=-P
# Temp dir.
tempDir=/cygdrive/C/Temp


# Help!
usage() {
cat << STOP_HELP
Usage: $0 --term searchTerm --file filePattern
Search set of files specified by filePattern for searchTerm.
          --file filePattern   Defines what file names to search for.
                               For example, "*.java" to search all Java files.
                               Can be a regex if you use -r.
                               This is not required if --type is used, but this will override
                               --type option if supplied.
           --term searchTerm   String to search for within files specified by file pattern.
  === Options that can be used. ===
                     [--c x]   x = number of lines to output on each side of each match (turns pretty off).
           [--dir searchDir]   Directory within which to look for files matching file pattern.
                               Defaults to current directory.
                    [--help]   Display help (this message) and exit.
               [--type type]   Specify type of files to search for. Shortcut for --file option.
                               Overridden by --file option. Available shortcut types:
                                  j for *.java files.
                                  p for *.jsp files.
                                  r for *.properties files.
                                  s for *.sql files.
                                  t for *.txt files.
                                  x for *.xml files.
                        [-f]   Output to a file which is then displayed in UltraEdit.
                        [-l]   Show matching file names only (turns pretty off).
                        [-L]   Follow symbolic links (as per find command).
                        [-n]   Avoid prettying up the output (automatic with -c and -l).
                        [-P]   Never follow  symbolic  links (as per find command). DEFAULT is this.
                        [-r]   --file filePattern is a regular expression, as per find: -iregex pattern
                        [-s]   Make search case sensitive.

Example usage 1.
    search -r --term 'folder' --file '.*.jsp?'
    Look for the term "folder" in js or jsp files.

STOP_HELP
}

# Process single hyphen arguments.
processBitArguments() {
   argument=$1
   while [ -n "${argument}" ]
   do
      nextArgument=${argument:0:1}
      argument="${argument:1}"
      case "${nextArgument}" in
         f) console=no;;
         l) fileNamesOnly=-l;;
         L) symbolic="-L";;
         n) pretty=no;;
         P) symbolic="-P";;
         r) filePatternRegularExpression=Y;;
         s) case=;;
         *) echo "*** Unknown arguments in -$1. ***"; usage; exit 3;;
      esac
   done
}

# Process all arguments.
if [ $# -lt 2 ] ; then
    echo "** Incorrect number of args specified **"; usage; exit 2
fi
while [ $# -gt 0 ]; do    # Until you run out of parameters . . .
   case "${1}" in
        "--c"   ) numberOfLines="$2"; shift;;
      "--dir"   ) if [ ! -d "${2}" ] ; then
                    echo "** Search directory ["${2}"] is not a dir or does not exist. **"; usage; exit 45
                  fi
                  searchDir="$2"; shift;;
      "--file"  ) filePattern="$2"; shift;;
      "--help"  ) usage; exit 0;;
      "--term"  ) searchTerm="$2"; shift;;
      "--type"  ) filePatternType="$2";shift;;
      -*        ) processBitArguments ${1/-/};;

   esac
   shift   # Check next set of parameters.
done

if [ -z "${filePattern}" -a -n "${filePatternType}" ] ; then
      case "${filePatternType}" in
         j) filePattern='*.java';;
         p) filePattern='*.jsp';;
         r) filePattern='*.properties';;
         s) filePattern='*.sql';;
         t) filePattern='*.txt';;
         x) filePattern='*.xml';;
         *) echo "*** Unknown --type: $filePatternType. ***"; usage; exit 3;;
      esac
fi


if [ -z "$filePattern" ]
then
   echo "*** File pattern not specified. ***"
   usage
   exit 4
elif [ -z "$searchTerm" ]
then
   echo "*** Search term not specified. ***"
   usage
   exit 5
fi


# Turn off pretty to show file names only.
if [ "$fileNamesOnly" = "-l" ]
then
   pretty=no
fi

# Turn off pretty to show number of lines
if [ "$numberOfLines" -ne 0 -o "$numberOfLines" -eq 0 ] 2> /dev/null
then
   # Don't pretty it up if number of lines is greater than 0
   if [ "$numberOfLines" -ne 0 ]
   then
      pretty=no
   fi
   numberOfLines="-C $numberOfLines"
else
   echo "[-c x] where x must be a positive integer, not: $numberOfLines"
   usage
   exit 21
fi


# Go to right dir and turn it into an absolute path.
oldDir=`pwd`
cd "$searchDir"
searchDir=`pwd`

# echo Search Term: "$searchTerm" > ${tempDir}/search.out.txt
# echo File Pattern: "$filePattern" >> ${tempDir}/search.out.txt
# echo Dir: "$3" >> ${tempDir}/search.out.txt
# echo "" >> ${tempDir}/search.out.txt

# if [ -f "$filePattern" ]
# then
#    files="$filePattern"
# else
#    files=`find . -type f -name "$filePattern"`
# fi
# if [ -z "$files" ]
# then
#    echo "No files found."
#    exit 2
# fi


if [ -f "$filePattern" ]
then
   files="$filePattern"
   grep ${case} "$numberOfLines" ${fileNamesOnly} -n "$searchTerm" $files > ${tempDir}/search.out.txt 2>&1
else
   if [ "$filePatternRegularExpression" = "Y" ]
   then
      find ${symbolic} . -type f -iregex "$filePattern" -print0 | xargs -0 grep ${case} "$numberOfLines" ${fileNamesOnly} -n "$searchTerm" | grep -v '[.]svn-base$\|zzbuild' > ${tempDir}/search.out.txt 2>/dev/null
   else
      find ${symbolic} . -type f -name "$filePattern" -print0 | xargs -0 grep ${case} "$numberOfLines" ${fileNamesOnly} -n "$searchTerm" | grep -v '[.]svn-base$\|zzbuild' > ${tempDir}/search.out.txt 2>/dev/null
   fi

fi

cd ${tempDir}/

# pretty it up?
if [ "$pretty" = "yes" ]
then
   # Replace first ":" with a new line..
   cat search.out.txt | sed -e 's/:/\
   /' -e 's/^[.]/\
   ./'  > /tmp/search.preprocessed.txt 2>&1

   echo Search for pattern \""$searchTerm"\" in dir $searchDir through file pattern \"$filePattern\"  2>&1 | tee search.processed.txt /tmp/search.processed.txt

   fileSet=
   fileIndex=0

   while read line
   do
      case $line in
          "") continue ;;
          ./*) # Is line a file name? (Will all filenames start with ./?)

              currentFile=$line
              if [ "$currentFile" != "$lastFile" ]
              then

                # store file name...
               fileSet[$fileIndex]="$line"
               let "fileIndex++"

               lastFile="$currentFile"
               printf "\n====\n%s\n" "$lastFile" >> search.processed.txt
               printf "\n====\n%s\n" "$lastFile"
              fi;;
           *) printf "%s\n" "$line" >> search.processed.txt
              printf "%s\n" "$line";;
      esac
   done < /tmp/search.preprocessed.txt

else
   cp search.out.txt search.processed.txt
   cat search.processed.txt
fi

if [ "$console" = "yes" ]
then
   if [ "${#fileSet[@]}" -gt 0 -a -n "${fileSet[0]}" ]
   then
      echo -e "\n\nDo you want to view any of the matching files?\n============"
       u -d "$searchDir" -i f ${fileSet[@]}
   fi
   echo
else
    # Open results up in favourite editor.
   U search.processed.txt
fi

cd "$oldDir"

Friday, December 12, 2008

Searching files within Vim and opening results with Vim

When I am programming, I often swap between using Eclipse or a Cywin + UltraEdit combination.

In Eclipse, Control+Shift+T is a wild card enabled search for Java files by file name and Control+Shift+R is a wild card enabled search for everything else (plus Java files) by file name - where you can double click on a result to open the file. You also have a file search (with regex) with Control+H.

In Cygwin, I use my own script to regex search for files by name, opening whichever file I need in UltraEdit. I also have a couple of scripts that grep through file contents and pass the result to the above script, which offers me a menu of the results and allows me to choose which of the results to open. I find this incredibly handy for many situations when I want to drop to a command line and search with a regular expression, or when I want a refreshing burst of geeky joy from using a command line.

I find it is easier to use my script on the command line to search through files and open the match I need than it is to use Eclipse's file search.

Recently, I have started to use Vim more and more - a particularly useful skill for times when I need to SSH to a remote *nix box and I don't have an UltraEdit to help me. Like any program, there is a learning curve, but text editors like Vim and Emacs have a steeper curve because there are no GUI menus to explore. However, the reward is arguably higher because as you learn how to use the app, you find so many ways to make your life easier.

I decided to try and match the functionality above using Vim. Here is the way I have found to allow searching for files by name and contents, seeing a list of results and easily opening one of the results.

Put the following in your .vimrc file.


" http://vim.wikia.com/wiki/Display_shell_commands'_output_on_Vim_window
" Run a shell command and open results in a horizontal split
command! -complete=file -nargs=+ Split call s:RunShellCommandInSplit(<q-args>)
function! s:RunShellCommandInSplit(cmdline)
  botright new
  setlocal buftype=nofile bufhidden=wipe nobuflisted noswapfile nowrap
  call setline(1,a:cmdline)
  call setline(2,substitute(a:cmdline,'.','=','g'))
  execute 'silent $read !'.escape(a:cmdline,'%#')
  setlocal nomodifiable
  1
endfunction

" Copy of the above that opens results in a new tab.
command! -complete=file -nargs=+ Tab call s:RunShellCommandInTab(<q-args>)
function! s:RunShellCommandInTab(cmdline)
  tabnew
  setlocal buftype=nofile bufhidden=wipe nobuflisted noswapfile nowrap
  call setline(1,a:cmdline)
  call setline(2,substitute(a:cmdline,'.','=','g'))
  execute 'silent $read !'.escape(a:cmdline,'%#')
  setlocal nomodifiable
  1
endfunction
" Issue a find command using regex and open results in a new tab.
command! -nargs=+ Find call s:RunShellCommandInTab('find . -iregex '.<q-args>)


With these two, I can now very easily run a shell command from within Vim and have the results appear in a new buffer - either a horizontal split or a new tab. For example here are three ways to run the shell command "ls -la" from within Vim:

:Split ls -la
:Tab ls -la
:! ls -la

:Split puts the results in a new horizontal split. :Tab puts the results in a new tab. The third technique is the built in way to run a shell command from within Vim. It hides Vim i.e. shows the command line, runs the command and shows the results in the command line and displays a prompt to go back to Vim: Press ENTER or type command to continue.

Now for the kicker. If I use :Split or :Tab, I can move the cursor to rest within whichever file I am interested in and press gf to open the file.

To find a file within a project's directory tree, ls -la isn't enough: I need the find command. The two examples below are equivalent.

:Find ".*\.css"
:Tab find . -iregex ".*\.css"

The first - much shorter - example uses the last short cut now in my .vimrc file. They both do a recursive regex search for a file through the entire directory tree, beginning from the directory I opened Vim from. Again, I put my cursor within whichever result I am interested in and press gf to open the file - which replaces the search results.

Always looking for shorter shortcuts for common searches, I put this small command in my bin dir: ~/bin/fjg ("fjg" for "find Java then grep").

#!/bin/bash
find . -name "*.java" | grep -i $@

It makes it much easier to search for Java source files quickly. Now I search with a shorter string:

:Tab fjg SomeClass

If I want to search through the contents of the files and select which result file to open, this is easy too:

:Find ".*\.css" | xargs grep -l span
:Tab find . -iregex ".*\.css" | xargs grep -l span

Here are some links that helped me write this.

Firefox plugin: It's All Text - make it use the same instance of GVim

One of my favourite Firefox plugins is It's All Text! which allows you to use an external editor to edit text areas in a browser page.

One of my favourite text editor (up there with UltraEdit) is vim and gvim. I wanted to use "It's All Text!" with gvim. It is easy enough to point the plugin at the gvim executable, but this meant each time I used the plugin, it would open a new gvim instance.

For Windows, this is the workaround I found to make sure "It's All Text!" would always open in the same gvim instance - opening a new tab each time. Create a cmd file:

@echo off
"C:\Program Files\Vim\vim72\gvim.exe" --remote-tab-silent %1

Then point "It's all Text" to that batch file instead, e.g. C:\Documents and Settings\rbram\My Documents\Tools\Windows\editFileInExistingVim.cmd

Tuesday, December 09, 2008

How to flush the cache with FatWire's ContentServer 5.5

Flush the cache with a URL like this:

http://DOMAIN/servlet/CacheServer?all=true&username=USERNAME&password=PASSWORD
If it worked, you should see a response like this: CacheServer flushing all pages.

Trouble shooting tips.

  • If flushing gives you this error: CacheServer : No access allowed for requested action.(Force page flush), check the following.
    • Are you sure your username and password are correct?
    • Does that user name have SiteGod ACL?
    • Are you logged in?
  • I made some changes and can't see them on refresh! Check your URL. If it is a satellite URL (http://DOMAIN/servlet/Satellite), change Satellite to ContentServer, i.e. http://DOMAIN/servlet/ContentServer and refresh again.
  • CSElements called by SiteEntry - to cache or not to cache.
    • If it is static (like some non changing content or maybe a Javascript file), cache it. In the SiteEntry, set "Pagelet Cache Criteria" to "true,*".
    • If the element is dynamic, i.e. it performs some processing and the output should be different each time, don't cache it. In the SiteEntry, set "Pagelet Cache Criteria" to "false".
    • See CacheInfo String Syntax section of the developer pdf for a detailed description of how to use this setting.
    • I said don't cache, but it is! I found (CS 5.5) that when I set my SiteEntry initially to "true,*" and changed it afterwards to "false", it was still being cached. To fix it, I deleted and re-created the SiteEntry.

Asset Descriptor Files in FatWire's Content Server 5.5

A few tips and things to watch for with Asset Descriptor Files (ADFs) under Content Server in the FatWire 5.5 Content Manaqement System (CMS). An ADF is the XML file used to define and create a basic asset using the AssetMaker utility. I am not sure how many of these issues are still relevant in newer versions.

General Tips

  • Keep PROPERTY NAME values all in lower case.
  • Keep INPUTFORM TYPE values all in upper case.

Date Fields

Here is a sample date field. Note that it allows searching for the date field.

<PROPERTY NAME="closedate" DESCRIPTION="Close Date">
   <STORAG TYPE="TIMESTAMP" LENGTH="20" />
   <INPUTFORM TYPE="TEXT" REQUIRED="NO" WIDTH="30"
         INSTRUCTION="Format the date as YYYY-MM-DD HH:MI:SS. Defines the date/time after which this is closed." />
   <SEARCHFORM DESCRIPTION="Close Date" TYPE="TEXT" WIDTH="30" MAXLENGTH="30" VERB="="/>
</PROPERTY>

Note the SEARCHFORM uses the = verb so that people search by exact date. Question: how to do date ranges?

Select Fields

Here is a sample select field. Note that it allows searching. Note that both search and input use string based values to populate the control as opposed to values drawn from the database.

<PROPERTY NAME="resultgraphtype" DESCRIPTION="Result Graph Type">
   <STORAGE TYPE="VARCHAR" LENGTH="10" />
   <INPUTFORM TYPE="SELECT" REQUIRED="NO" DEFAULT="hor bar"
         INSTRUCTION="Defines what graph type (if any) will be used for display."
         SOURCETYPE="STRING"
         OPTIONDESCRIPTIONS="Horizontal Bar Graph,Vertical Bar Graph,Pie Graph,Ranking Chart"
         OPTIONVALUES="hor bar,ver bar,pie,rank"
   <SEARCHFORM TYPE="SELECT" DESCRIPTION="Result Graph Type"
         SOURCETYPE="STRING"
         OPTIONDESCRIPTIONS="Horizontal Bar Graph,Vertical Bar Graph,Pie Graph,Ranking Chart"
         OPTIONVALUES="hor bar,ver bar,pie,rank" />
</PROPERTY>

Element Property Types

On DEV (CS 5.5) this worked:

<PROPERTY NAME="options" DESCRIPTION="Options">
   <STORAGE TYPE="VARCHAR" LENGTH="1"/>
   <INPUTFORM TYPE="ELEMENT" REQUIRED="NO"
         INSTRUCTION="Fill out potential options."/>
</PROPERTY>

Trouble Shooting

First steps.

  • Open the XML ADF in a browser like Firefox, which will parse the XML and give you a bit more information about malformed XML.
  • If a browser like Firefox opens the ADF ok, it should be properly formed XML, which still means it might be breaking a FatWire specific parsing rule i.e. it is valid XML, but not not valid ADF XML. Look at the FatWire documentation on the error numbers for some further clues.
  • If you are getting a problem with your ADF and you are unable to nail down which property it relates to, start removing properties from your ADF and re-loading it until you find a point at which the error no longer occurs. This means: delete the asset and table from Content Server, modify the ADF to remove one property, re-make the asset, register asset elements and create the table.

A Short FAQ on Common Problems

  • -2002 processing asset descriptor file. AssetMaker: Descriptor parse failed.
    • There were some problems with my INPUTFORM elements. If you find a different solution, please let me know!
      • I was missing some closing "/>"s.
      • My INSTRUCTION attributes too long - they have a limit of 80 characters.
  • AssetMaker: An error occurred creating the asset table. Please check the descriptor file for invalid STORAGE tags. (AssetMaker error number=-2005, error number=-100)
    • There was an error with a STORAGE element. If you find a diferent solution, please let me know!
      • <STORAGE TYPE="varchar" needed to be <STORAGE TYPE="VARCHAR" --- just capitalize VARCHAR!!!
  • Do you find that your text fields get pre-populated with contents such as Variables.ContentDetails:mandatoryMessage?
      Make sure your property names are all lower case and have no underscores. For example, do this:
      <PROPERTY NAME="mandatorymessage" DESCRIPTION="Mandatory Message">
      instead of this
      <PROPERTY NAME="mandatoryMessage" DESCRIPTION="Mandatory Message">
      or this
      <PROPERTY NAME="mandatory_message" DESCRIPTION="Mandatory Message">
      Source of this answer from the FatWire forum.
  • Could not delete a basic asset.
    • The error was: This attempt to delete the asset failed with error -105. The most likely cause is a system configuration error. Please exit the browser, login, and then try the operation again. If the problem persists, please contact your System Administrator.
    • The issue was a STORAGE TYPE that had to be changed from CHAR to VARCHAR. Here is the changed property.
      <PROPERTY NAME="multivoting" DESCRIPTION="Multiple Voting">
         <STORAGE TYPE="VARCHAR" LENGTH="3"/>
         <INPUTFORM REQUIRED="NO" TYPE="RADIO" DEFAULT="No"
               SOURCETYPE="STRING" RBDESCRIPTIONS="Yes,No" RBVALUES="Yes,No"
               INSTRUCTION="Define whether users are able to vote more than once."/>
         <SEARCHFORM TYPE="RADIO" DESCRIPTION="Multiple Voting"
               SOURCETYPE="STRING" RBDESCRIPTIONS="Yes,No" RBVALUES="Yes,No" />
      </PROPERTY>