Dynamically generating Zip files using Google Cloud Storage Client Library for Appengine

Let's say you have 50 objects (15Mb each) stored in Google Cloud Storage. Now, you need to create a zip archive containing all of them and store the resultant file back into GCS. How can we achieve that from within an Appengine java application?
Well, after some research, I wrote the method below using Google Cloud Storage Client Library which does exactly that. Just don't forget  to grant the appropriate permissions to your appengine service account so that it can read and write the objects.

public static void zipFiles(final GcsFilename targetZipFile,
  final GcsFilename... filesToZip) throws IOException {

 Preconditions.checkArgument(targetZipFile != null);
 Preconditions.checkArgument(filesToZip != null);
 Preconditions.checkArgument(filesToZip.length > 0);

 final int fetchSize = 4 * 1024 * 1024;
 final int readSize = 2 * 1024 * 1024;
 GcsOutputChannel outputChannel = null;
 ZipOutputStream zip = null;
 try {
  final GcsFileOptions options = new GcsFileOptions.Builder()
    .mimeType(MediaType.ZIP.toString()).build();
  outputChannel = GCS_SERVICE.createOrReplace(targetZipFile, options);
  zip = new ZipOutputStream(Channels.newOutputStream(outputChannel));
  GcsInputChannel readChannel = null;
  for (final GcsFilename file : filesToZip) {
   try {
    final GcsFileMetadata meta = GCS_SERVICE.getMetadata(file);
    if (meta == null) {
     LOGGER.warning(file.toString()
       + " NOT FOUND. Skipping.");
     continue;
    }
    final ZipEntry entry = new ZipEntry(file.getObjectName());
    zip.putNextEntry(entry);
    readChannel = GCS_SERVICE.openPrefetchingReadChannel(file,
      0, fetchSize);
    final ByteBuffer buffer = ByteBuffer.allocate(readSize);
    int bytesRead = 0;
    while (bytesRead >= 0) {
     bytesRead = readChannel.read(buffer);
     buffer.flip();
     zip.write(buffer.array(), buffer.position(),
       buffer.limit());
     buffer.rewind();
     buffer.limit(buffer.capacity());
    }

   } finally {
    zip.closeEntry();
    readChannel.close();
   }
  }
 } finally {
  zip.flush();
  zip.close();
  outputChannel.close();
 }
}

"Smart" Appengine Devserver restarts for faster development lifecycle

We just started a new AppEngine Java project using the appengine-maven-plugin and the Eclipse IDE (Juno). We are using the appengine:devserver goal to start the devserver. It basically builds the entire project (compile, test and package) and after that launches the devserver pointing it to the generated webapp directory - which by default is: ${project.build.directory}/${project.build.finalName}

The Problem

Every edit made in the source directory is not recognized unless the server is restarted - neither static content nor newly compiled classes. Which is obvious since the devserver is monitoring only the target directory.
It's a very "bureaucratic" and nonproductive development environment - we need to stop and start the server even for a single CSS line change.

The Dream

Achieve the same productivity level we have when working with dynamic languages based development environment (i.e. Python). Just hit F5 in the browser to see the changes in static files and automatic server reload every time a Java class or descriptor file is compiled/changed.

The Solution 

Using a little Ant-foo, we were able to create a target which synchronizes both directories.
Thie snippet below uses the sync Ant task to perform the static content synchronization (lines 1-9). Notice that everything inside src.webapp.dir is sync'ed except for the 3 directories declared in the preserveintarget element. We had to exclude them from the synchronization process because since they only exist in the target directory they'd be deleted otherwise. And finally a second sync to synchronize the compiled classes (lines 11-13).

<sync verbose="true" todir="${target.webapp.dir}" includeEmptyDirs="true">
 <fileset dir="${src.webapp.dir}" />
 <preserveintarget>
     <!-- Ignore the directories below -->
  <include name="WEB-INF/lib/**" />
  <include name="WEB-INF/classes/**" />
  <include name="WEB-INF/appengine-generated/**" />
 </preserveintarget>
</sync>

<sync verbose="true" todir="${target.webapp.dir}/WEB-INF/classes">
 <fileset dir="${basedir}/target/classes" />
</sync>

Then, we attached it to an Eclipse builder, which is triggered every time a change is made in the project ("Build automatically" flag enabled).
The same behavior can be achieved by creating a special maven profile and using a combination of the m2e lifecycle mappings and the maven antrun plugin.
Something like this:

<profile>
    <id>m2e</id>
    <activation>
        <property>
            <name>m2e.version</name>
        </property>
    </activation>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-antrun-plugin</artifactId>
                <version>1.7</version>
                <executions>
                    <execution>
                        <phase>process-classes</phase>
                        <goals>
                            <goal>run</goal>
                        </goals>
                        <configuration>
                            <target>
                                <property name="target.webapp.dir" value="${project.build.directory}/${project.build.finalName}" />
                                <property name="src.webapp.dir" value="${basedir}/src/main/webapp" />
                                <sync verbose="true" todir="${target.webapp.dir}" includeEmptyDirs="true">
                                    <fileset dir="${src.webapp.dir}" />
                                    <preserveintarget>
                                        <include name="WEB-INF/lib/**" />
                                        <include name="WEB-INF/classes/**" />
                                        <include name="WEB-INF/appengine-generated/**" />
                                    </preserveintarget>
                                </sync>
                                <sync verbose="true" todir="${target.webapp.dir}/WEB-INF/classes">
                                    <fileset dir="${basedir}/target/classes" />
                                </sync>
                            </target>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
        <pluginManagement>
            <plugins>
                <!-- This plugin's configuration is used to store Eclipse m2e settings 
      only. It has no influence on the Maven build itself. -->
                <plugin>
                    <groupId>org.eclipse.m2e</groupId>
                    <artifactId>lifecycle-mapping</artifactId>
                    <version>1.0.0</version>
                    <configuration>
                        <lifecycleMappingMetadata>
                            <pluginExecutions>
                                <pluginExecution>
                                    <pluginExecutionFilter>
                                        <groupId>org.apache.maven.plugins</groupId>
                                        <artifactId>maven-antrun-plugin</artifactId>
                                        <versionRange>[1.6,)</versionRange>
                                        <goals>
                                            <goal>run</goal>
                                        </goals>
                                    </pluginExecutionFilter>
                                    <action>
                                        <execute>
                                            <runOnIncremental>true</runOnIncremental>
                                        </execute>
                                    </action>
                                </pluginExecution>
                            </pluginExecutions>
                        </lifecycleMappingMetadata>
                    </configuration>
                </plugin>
            </plugins>
        </pluginManagement>
    </build>
</profile>