With an S3 API you'll be uploading to the server, then uploading to S3. If you don't need any post-processing, you shouldn't be hitting your web server in the first place and uploading directly to S3 (using CORS).Īssuming you are hitting the server probably means you need to do some post-processing on images. If you don't need any post-processing (thumbnail generation for example). Most of your S3 requests are going to be PUT (~5%) and GET (~95%). The best thing about S3FS is you have one less things to worry about and get some performance benefits for free. Initially, it had a number of bugs and memory leaks (I had a cron-job to restart it every 2 hours) but with the latest release 1.73 it's been very stable. This is an old question so I'll share my experience over the past year with S3FS. This is a risk I was able to live with and was the option I chose in the end. But you should know that although the most performant option it has one big problemĪn EBS Mounted NFS Share has its own problems - a single point of failure if the machine that's sharing the EBS Volume goes down then you lose access on all machines which access the share. I resorted to EBS Mounted Drived shared from an EC2 instance. Will probably want to increase block size. With the default 4K block size that's not an issue but most users memory usage can be prohibitive: by default it caches 1000 blocks. ![]() too large block sizes can add significant data transfer and storage.too small block sizes (e.g., the 4K default) can add significantĮxtra costs (e.g., $130 for 50GB with 4K blocks worth of storage).High risk for data corruption, due to the delayed writes.This gets around the performance issues but itself has a few issues of its own: ![]() The article mentioned above does talk about a similar application - s3backer - which gets around the performance issues by implementing a virtual filesystem over S3. If you're talking about appliation data, (say database files, logging files) where you want to make small incremental change then its a definite no - S3 Just doesn't work that way you can't incrementally change a file. If you're storing say, photos, where you want to write an entire file or read an entire file never incrementally change a file, then its fine, although one may ask, if you're doing this, then why not just use S3's API Directly? It therefore depends on what you are storing whether s3fs is a feasible option.
0 Comments
Leave a Reply. |