[tbnl-devel] File Upload Design Issues

[Jeff Caldwell]

The code Edi referred to is only a proof of concept at
the moment. Don't bother looking at it yet. (See
below.) It did let me upload a file. I'm happy to look
at it again, though. There are things I need to
research, and I am asking for everyone's help and
advice so that I get the code as right as possible:

1) What are the relevant RFC's I need to consider?

2) What references/URLs/sample code is available for
me to learn about the encoding/stream issues?

3) General comments on the design. In particular,
uploading a file may only the first step. What is to
be done with the file? 

3a) There are OS dependencies, as not all OS's support
the same file names. 

3b) The file upload must generate a portable but
unique internal name while at the same time retaining
the OS-specific name specified by the sender (original
name). 

3c) This implies a kind of catalog of files, mapping
the original name to the internal name and vice versa.
This technique also is necessary to prevent duplicate
file names, for example from multiple users. 

3d) The user should be able to associate a short
and/or long description, and possibly a category or
two, with the file. TBNL might not dictate this
precisely but the catalog should be flexible in what
it holds.

3e) The catalog needs to be able to associate a file
with a user and vice versa. Users aren't a TBNL
concept at the moment and I'm not sure adding them
just for this is worthwhile. On the other hand,
uploading and storing files without the possibility of
a sense of ownership doesn't seem robust. Is this
really a part of the TBNL core? How can this be
handled cleanly?

3f) Where and how should the files be stored? ACS and
OpenACS (openacs.org), the last time I looked, store
uploaded files in a big directory tree with, IIRC,
subdirectories named with single letters. The
destination subdirectory was chosen by the first 3 or
4 chars of the internally-generated file name. The
idea is that a filesystem can have, or can efficiently
handle, only so many files per subdirectory. Given an
idea of the total number of uploaded files we want to
support, we could calculate the depth and number of
subdirectories needed.

3g) TBNL-level security is enough and it's OK to
mix uploaded files from several users in the same
catalog and subdirectories, so long as TBNL keeps
things sorted out and, depending upon the application,
the programmer is responsible for showing/making
available the right file to the right person.

I bet I both left something out and put too much in :)
I'd appreciate comments on all of the above. In
summary, how should the files be stored and how should
the catalog be structured? Once I have an idea of how
we think this should work, I'm happy to glue it all
together. Thanks for your help.

Jeff


--- Edi Weitz <edi at agharta.de> wrote:

> On Mon, 30 Aug 2004 17:19:17 -0500, Travis Cross
> <travis at crosswirecorp.com> wrote:
> 
> > I have one question that I haven't seen come up on
> the mailing list
> > or in the documentation: what are the hurdles or
> considerations in
> > implementing multipart form / file upload handling
> in TBNL ?
> 
> See Stefan's reply. My main problem is time. I
> haven't even found the
> time to read through all the relevant RFCs. I also
> think that streams
> (changing the encoding on the fly) might be an issue
> here.
> 
> This feature will probably have to wait until I need
> it in a project
> or until someone else provides a working patch.
> 
> You might want to look at Jeff Caldwell's code at
> tbnl.org - I seem to
> remember that he had started working on this. Jeff?
> 
> Cheers,
> Edi.
> 


		
_______________________________
Do you Yahoo!?
Win 1 of 4,000 free domain names from Yahoo! Enter now.
http://promotions.yahoo.com/goldrush