Saturday, December 13, 2008

Cygwin Bash script: search through files and let user open results

I frequently use this Bash script in Cgywin to easily search for a file by name (using regex), see a list of results and choose which of those results I want to open up in my favourite text editor, like UltraEdit. The script in this post my own wrapper for grep and find that allows me to search through the contents of files, show the results of the search and call the previous script to show me a list of matching files, asking me which ones I want to open.

Purpose. Search through files, show user results and allow user to open any of the files that contained a match.

Input. File name search criteria, file contents search criteria, selection of what files to output.

Output. Lines of results, sorted by file, menu of result files to open.

Dependencies. Relies on this Bash script being present in the path - refers to it as u - because when I first wrote it, I would only use it to open files in UltraEdit!

Example 1.

  1. Search through all *.java files for the regular expression setco.*nager.
  2. Display results sorted by file.
  3. Allow user to see a list of result files and select which ones, if any, to open.
-bash3.2 - rbram@MCLL - Javascript TOC - /cygdrive/c/temp/DocPublisher
Sat Dec 13 - 08:13 PM > search --type j --term "setco.*nager"
Search for pattern "setco.*nager" in dir /cygdrive/c/temp/DocPublisher through file pattern "*.java"

====
./src/java/au/com/publisher/doc/PublisherServiceImpl.java
846:    public void setCollectionManager(

====
./test/java/other/au/com/publisher/admin/MockData.java
1005:           publisherServiceImpl.setCollectionManager(mockCollectionManager);

Do you want to view any of the matching files?
============
File  0: ./src/java/au/com/publisher/doc/PublisherServiceImpl.java
File  1: ./test/java/other/au/com/publisher/admin/MockData.java
Specify files to open. [A]ll, [N]one or [x y z] space separated indexes.

Code

#!/bin/bash

#-------------------------------------------------------------------------------
#  Search Command
#-------------------------------------------------------------------------------
# Search through files in any location and open any of them in an editor.
#
# Author: Robert Mark Bram
# v 1.2 - 1920/11/2007 1:37:05 PM. Included the ability to follow (or not) symbolic
#   links by hooking into the find command's capability for this. By default, do
#   not follow symbolic links.
# v 1.3 - 29/02/2008 4:03:53 PM. Don't look in files matching:
#   grep -v '[.]svn-base$\|zzbuild'

#-------------------------------------------------------------------------------
#  Dependencies
#-------------------------------------------------------------------------------
# - Relies on the 'u' command.

#-------------------------------------------------------------------------------
#  Variables for this script.
#-------------------------------------------------------------------------------
# Variables for this script.
# Pretty up the output to make it more readable. Yes or no.
pretty=yes
# Make the search case sensitive (-i) or not (no value).
case=-i
# Output results to console. Yes or no.
console=yes
# Pattern of files in which we shall search.
filePattern=
# Output names of files only.
fileNamesOnly=
# Number of lines around the search result to display.
numberOfLines=0
# Directory within which to search.
searchDir=.
# Was --type given to specify a file type? This var will have that value.
filePatternType=
# Should filePattern be treated as a regular expression as per find: -iregex pattern?
filePatternRegularExpression=N
# Follow symbolic links? -P (no) or -L (yes) - as per the find command.
symbolic=-P
# Temp dir.
tempDir=/cygdrive/C/Temp


# Help!
usage() {
cat << STOP_HELP
Usage: $0 --term searchTerm --file filePattern
Search set of files specified by filePattern for searchTerm.
          --file filePattern   Defines what file names to search for.
                               For example, "*.java" to search all Java files.
                               Can be a regex if you use -r.
                               This is not required if --type is used, but this will override
                               --type option if supplied.
           --term searchTerm   String to search for within files specified by file pattern.
  === Options that can be used. ===
                     [--c x]   x = number of lines to output on each side of each match (turns pretty off).
           [--dir searchDir]   Directory within which to look for files matching file pattern.
                               Defaults to current directory.
                    [--help]   Display help (this message) and exit.
               [--type type]   Specify type of files to search for. Shortcut for --file option.
                               Overridden by --file option. Available shortcut types:
                                  j for *.java files.
                                  p for *.jsp files.
                                  r for *.properties files.
                                  s for *.sql files.
                                  t for *.txt files.
                                  x for *.xml files.
                        [-f]   Output to a file which is then displayed in UltraEdit.
                        [-l]   Show matching file names only (turns pretty off).
                        [-L]   Follow symbolic links (as per find command).
                        [-n]   Avoid prettying up the output (automatic with -c and -l).
                        [-P]   Never follow  symbolic  links (as per find command). DEFAULT is this.
                        [-r]   --file filePattern is a regular expression, as per find: -iregex pattern
                        [-s]   Make search case sensitive.

Example usage 1.
    search -r --term 'folder' --file '.*.jsp?'
    Look for the term "folder" in js or jsp files.

STOP_HELP
}

# Process single hyphen arguments.
processBitArguments() {
   argument=$1
   while [ -n "${argument}" ]
   do
      nextArgument=${argument:0:1}
      argument="${argument:1}"
      case "${nextArgument}" in
         f) console=no;;
         l) fileNamesOnly=-l;;
         L) symbolic="-L";;
         n) pretty=no;;
         P) symbolic="-P";;
         r) filePatternRegularExpression=Y;;
         s) case=;;
         *) echo "*** Unknown arguments in -$1. ***"; usage; exit 3;;
      esac
   done
}

# Process all arguments.
if [ $# -lt 2 ] ; then
    echo "** Incorrect number of args specified **"; usage; exit 2
fi
while [ $# -gt 0 ]; do    # Until you run out of parameters . . .
   case "${1}" in
        "--c"   ) numberOfLines="$2"; shift;;
      "--dir"   ) if [ ! -d "${2}" ] ; then
                    echo "** Search directory ["${2}"] is not a dir or does not exist. **"; usage; exit 45
                  fi
                  searchDir="$2"; shift;;
      "--file"  ) filePattern="$2"; shift;;
      "--help"  ) usage; exit 0;;
      "--term"  ) searchTerm="$2"; shift;;
      "--type"  ) filePatternType="$2";shift;;
      -*        ) processBitArguments ${1/-/};;

   esac
   shift   # Check next set of parameters.
done

if [ -z "${filePattern}" -a -n "${filePatternType}" ] ; then
      case "${filePatternType}" in
         j) filePattern='*.java';;
         p) filePattern='*.jsp';;
         r) filePattern='*.properties';;
         s) filePattern='*.sql';;
         t) filePattern='*.txt';;
         x) filePattern='*.xml';;
         *) echo "*** Unknown --type: $filePatternType. ***"; usage; exit 3;;
      esac
fi


if [ -z "$filePattern" ]
then
   echo "*** File pattern not specified. ***"
   usage
   exit 4
elif [ -z "$searchTerm" ]
then
   echo "*** Search term not specified. ***"
   usage
   exit 5
fi


# Turn off pretty to show file names only.
if [ "$fileNamesOnly" = "-l" ]
then
   pretty=no
fi

# Turn off pretty to show number of lines
if [ "$numberOfLines" -ne 0 -o "$numberOfLines" -eq 0 ] 2> /dev/null
then
   # Don't pretty it up if number of lines is greater than 0
   if [ "$numberOfLines" -ne 0 ]
   then
      pretty=no
   fi
   numberOfLines="-C $numberOfLines"
else
   echo "[-c x] where x must be a positive integer, not: $numberOfLines"
   usage
   exit 21
fi


# Go to right dir and turn it into an absolute path.
oldDir=`pwd`
cd "$searchDir"
searchDir=`pwd`

# echo Search Term: "$searchTerm" > ${tempDir}/search.out.txt
# echo File Pattern: "$filePattern" >> ${tempDir}/search.out.txt
# echo Dir: "$3" >> ${tempDir}/search.out.txt
# echo "" >> ${tempDir}/search.out.txt

# if [ -f "$filePattern" ]
# then
#    files="$filePattern"
# else
#    files=`find . -type f -name "$filePattern"`
# fi
# if [ -z "$files" ]
# then
#    echo "No files found."
#    exit 2
# fi


if [ -f "$filePattern" ]
then
   files="$filePattern"
   grep ${case} "$numberOfLines" ${fileNamesOnly} -n "$searchTerm" $files > ${tempDir}/search.out.txt 2>&1
else
   if [ "$filePatternRegularExpression" = "Y" ]
   then
      find ${symbolic} . -type f -iregex "$filePattern" -print0 | xargs -0 grep ${case} "$numberOfLines" ${fileNamesOnly} -n "$searchTerm" | grep -v '[.]svn-base$\|zzbuild' > ${tempDir}/search.out.txt 2>/dev/null
   else
      find ${symbolic} . -type f -name "$filePattern" -print0 | xargs -0 grep ${case} "$numberOfLines" ${fileNamesOnly} -n "$searchTerm" | grep -v '[.]svn-base$\|zzbuild' > ${tempDir}/search.out.txt 2>/dev/null
   fi

fi

cd ${tempDir}/

# pretty it up?
if [ "$pretty" = "yes" ]
then
   # Replace first ":" with a new line..
   cat search.out.txt | sed -e 's/:/\
   /' -e 's/^[.]/\
   ./'  > /tmp/search.preprocessed.txt 2>&1

   echo Search for pattern \""$searchTerm"\" in dir $searchDir through file pattern \"$filePattern\"  2>&1 | tee search.processed.txt /tmp/search.processed.txt

   fileSet=
   fileIndex=0

   while read line
   do
      case $line in
          "") continue ;;
          ./*) # Is line a file name? (Will all filenames start with ./?)

              currentFile=$line
              if [ "$currentFile" != "$lastFile" ]
              then

                # store file name...
               fileSet[$fileIndex]="$line"
               let "fileIndex++"

               lastFile="$currentFile"
               printf "\n====\n%s\n" "$lastFile" >> search.processed.txt
               printf "\n====\n%s\n" "$lastFile"
              fi;;
           *) printf "%s\n" "$line" >> search.processed.txt
              printf "%s\n" "$line";;
      esac
   done < /tmp/search.preprocessed.txt

else
   cp search.out.txt search.processed.txt
   cat search.processed.txt
fi

if [ "$console" = "yes" ]
then
   if [ "${#fileSet[@]}" -gt 0 -a -n "${fileSet[0]}" ]
   then
      echo -e "\n\nDo you want to view any of the matching files?\n============"
       u -d "$searchDir" -i f ${fileSet[@]}
   fi
   echo
else
    # Open results up in favourite editor.
   U search.processed.txt
fi

cd "$oldDir"