How can I persist my Data

[pstrotmann]

Hi,

I have deployed a web application with its database (DB4O) integrated in the application itself as a file in WEB-INF directory.
The application works fine. After the deployment I can enter Data into may integrated database and the data are persistent as long as I do not stop the deployment.
But when I have stopped the deployment and started tehe deployment again all data are lost.

What did I wrong ?

Peter

[dvolk]

Hi Peter,

When you stop a deployment, the instance(s) running your app are stopped and killed. So when you start a stopped deployment, AWS provisions new instances for you.

To persist your data you can use Elastic Block Storage (EBS) volumes: http://aws.amazon.com/ebs/. In CloudFoundry we use this functionality for MySQL database storage (see Storage Options when on the "Deploy Application" screen). If you don't need a MySQL database, and even if you do, you can store your DB4O file(s) there. Or, alternatively, you can create your own EBS volume via ElasticFox or AWS Management Console, and write a simple script to mount the volume to your instance.

Please let us know if you need help with that.

Thank you,

[pstrotmann]

Hi dvolk

Now that I have my application running, I have created a 1 GB volume on amazon ebs.
What steps are needed to bring my db4o dataset to that volume and acces that dataset from my application ?
On my local linux system I access the file via a pathname: /opt/db4o/myfile.yap
What pathname is needed for the dataset on the ebs volume ?

peter

[dvolk]

Hello Peter,

Using your own EBS volume (created via AWS console or ElasticFox)
To start using an EBS volume with your application you need to do the following:

1. Initially you need to format your volume and copy your data file(s) on it
2. Run a script to mount this volume every time you start or re-start a deployment.

1. To format your volume you would need to SSH into a running EC2 instance (it has to be in the same availability zone as the volume) and do the following:

$ ec2-attach-volume -d /dev/sdh -i <instance id> <volume id>
$ mkfs.xfs /dev/sdh

Then you need to mount the newly formatted volume:

$ mkdir /mnt/ebs
$ mount -t xfs /dev/sdh /mnt/ebs

After that you can copy/move your data file(s) to a location under /mnt/ebs

2. To automatically mount the volume every time, you can have a script like the following:

#!/bin/bash
if [ ! -a '/mnt/ebs' ] 
then
mkdir /mnt/ebs
fi

ec2-attach-volume -d /dev/sdh -i `curl -s http://169.254.169.254/latest/meta-data/instance-id` <volume id>

mount -t xfs /dev/sdh /mnt/ebs

You can then upload this script into the "App Server Startup Script" section of the "Create Deployment" screen in CloudFoundry.

Using CloudFoundry storage options
It could be easier to use CloudFoundry storage options. Although you don't need a MySQL database, you can still take advantage of managing EBS volumes that comes with it.

When you create a deployment, select "New EBS Volume" in "Database Storage" drop-down under the "Database" section. Specify the volume size. CloudFoundry will automatically create an EBS Volume for you and take care of attaching it every time you deploy or re-deploy your application. The volume is mounted under /vol, and you could place your data file(s) there.

Depending on your selection of "Backup frequency" you data could be backed up automatically along with the empty MySQL database via EBS Snapshots.

[pstrotmann]

HI dvolk,

Using CloudFoundry storage options seems the best option for me due to the automatism it offers.

thanks a lot

peter

[pstrotmann]

Hi Dmitriy,

Now that I have created a volume, I try to copy my db4o file to that volume:

I tried the following:

scp -i /home/peter/Downloads/amazon/cloudfoundry_eu-west-1.pem /vol/db4o/mitglieder.yap root@ip-10-226-234-100.eu-west-1.compute.internal:/vol

and received:

ssh: connect to host ip-10-226-234-100.eu-west-1.compute.internal port 22: Connection refused

Peter

[dvolk]

Peter,

You're using an internal DNS name of your instance, which doesn't resolve outside of EC2. Find your deployment in CloudFoundry web interface, click on it, then click on your instance, and in the instance details page you'll see External DNS name. Use that.

[pstrotmann]

Hi Dmitriy,

I copied my Dataset mitglieder.yap to /vol and was able to access to that dataset.

Because of a db4o problem, that I already posted in db4o forum, I can use the yap-file only if it is inside the webroot directory. So I tested a workaround on my local machine copying from vol to webroot on start of tomcat and copying back on shutdown. That works fine on my local machine, but on cloud foundry it seems that only the /vol to webroot direction ist working, so all modifications are lost.

I've built in some logging, that I can see on the tomcat console on my local machine, Can I make the tomcat logging visible on cloud Foundry too ?

Peter

[dvolk]

Hi Peter,

If you click on your deployment, and then on the instance that's running tomcat, on the instance details page there will be a button "View Log" under the "App Server Tier" section. Clicking on it will display your Tomcat log in a new browser window.

[pstrotmann]

Hi Dmitriy

tomcat log says:

java.io.FileNotFoundException: /vol/mitglieder.yap (Permission denied)

What permission is needed and how to set it ?

Peter