<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<META NAME="Generator" CONTENT="MS Exchange Server version 6.5.6944.0">
<TITLE>Suggested "filesystem" (was RE: Topic of list)</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/plain format -->
<P><FONT SIZE=2>-- Frode Vatvedt Fjeld wrote --<BR>
<BR>
>> Would enforcing a strict barrier between packages be enough to<BR>
>> solve package conflicts and problems, and what would this barrier<BR>
>> look like?<BR>
><BR>
> I believe there is only one real way to enforce safety and security<BR>
> barriers, and that's using the (page-based) memory access and<BR>
> privileges. Packages is mostly a naming mechanism, I think, so I<BR>
> don't think it's where one can deal with security.<BR>
<BR>
I didn't mean to confuse the packaging question with the security<BR>
question--I understand that basic security must go much deeper than a<BR>
package system. Still, I think having the package system being<BR>
responsible for much of the higher-level security wouldn't be a bad<BR>
thing (see last paragraph). Namespaces can be a powerful method of<BR>
organization, with more possible applications than just naming things.<BR>
<BR>
I don't see anything wrong with the normal kernel ideas of page-based<BR>
memory access privileges, but this isn't what the system administrator<BR>
sees. I guess my question was more oriented toward user-level<BR>
security, as in whether a multi-user OS applies to Lisp-OS's, whether<BR>
it's a good idea at all.<BR>
<BR>
> Personally I'd be willing to consider tossing out the traditional<BR>
> file-system, but I don't really have any great ideas as to what<BR>
> exactly the alternative would be for general purpose storage.<BR>
<BR>
I think the technical barriers to creating a true POS system might be<BR>
too great to try to provide such a thing right away. As James said,<BR>
it's not a well-understood problem. However, I think there are a lot<BR>
of features of a less-hierarchically-oriented system that could be<BR>
included without too much fuss.<BR>
<BR>
Here's what I was thinking. I'm a long-time Windows user. I write C#<BR>
software for a living that runs MS SQL Server and depends on Active<BR>
Directory for security. So I know Windows pretty well. I've been using<BR>
Linux for about three years now, and I've tried several distributions,<BR>
so I think my impressions of Linux are fairly unbiased toward Linux<BR>
itself, as opposed to some part of Linux. What I like about Windows is<BR>
that when you install things, you don't *have* to configure them.<BR>
Things get put where they should be, automatically. As far as I<BR>
understand, this is impossible in Unix-based OS's because there are no<BR>
standards to dictate where things ought to go, so a lot of effort is<BR>
put into figuring out where to put things and then telling everything<BR>
where everything else is. Correct me if I'm wrong, but this is what<BR>
makes it impossible to write installers for anything less specific<BR>
than a particular proprietary Unix or Linux distro running on a<BR>
particular hardware platform, in the best case.<BR>
<BR>
Let me fantasize about a better OS for a second. Let me call this OS<BR>
SOS, for "Super" OS. In SOS, anything you install has a name given to<BR>
it by its authors. For instance, cl-ppcre, the CL regex package. If<BR>
you wanted this package, you'd tell SOS the URL of the project, or<BR>
you'd give it a short name that it matched from a list of package<BR>
repository sites, and SOS downloads it and installs it, like<BR>
asdf-install, except that the package's files aren't put into some<BR>
directory somewhere, but instead into a big list of the system's<BR>
packages, maintained by the OS.<BR>
<BR>
The SOS filesystem (yes, it's a filesystem) is not /basically/<BR>
hierarchical, but flat. Each package has its own base directory, which<BR>
can have sub-directories, and each directory is given a number of<BR>
attributes that specify what it is. The base directory for a package<BR>
is given the attr-package attribute (maybe a psuedo-attribute, because<BR>
it shouldn't be neccesary to actually store this), and possibly the<BR>
attr-home-directory attribute; The user's files should not be mixed<BR>
with other packages. The directory where the binaries are stored<BR>
(located under the package's base directory) are given an attribute<BR>
that lets SOS know to put the files contained (or perhaps the exports<BR>
defined in the files contained) into its search path for commands. Or<BR>
this attribute could be applied to individual files in the package.<BR>
Any logs (say, for a web server package) would be in a directory<BR>
marked by the attr-log attribute, and so on.<BR>
<BR>
Naming conflicts would be resolved in this manner: each directory<BR>
/must/ be in a particular package, and each package has both a short<BR>
name, a description, and a GUID. The GUID is used by all packages<BR>
referencing the other package, and used consistently by the system,<BR>
internally. The short name of a package can be changed by the user, so<BR>
if they wanted to install two packages called "firebird" they could<BR>
refer to one as "firebird-browser" and the other as "firebird-db".<BR>
They could rename any package at any time without breaking anything.<BR>
Or, they could leave two packages names the same. The description is<BR>
used by the user to disambiguate if more than one short name is the<BR>
same, and the GUID by the system.<BR>
<BR>
The version of a package would be separate from the GUID, but part of<BR>
the dependency resolving process. Multiple versions of a package could<BR>
be installed side by side, and applications that depend on a package<BR>
could either specify the exact version they require, a minimum<BR>
version, a list of versions, or something else in that vein. If the<BR>
required version isn't installed, it would be downloaded and installed<BR>
automatically (with the user's permission, of course). The user could<BR>
override a package's dependency resolution rules in case of a failure<BR>
of a certain package provider to provide backwards compatibility, or<BR>
any number of other exceptional conditions.<BR>
<BR>
Navigation through the filesystem/package-system is sort of like<BR>
querying a relational DB. As complicated as that may first impress one<BR>
as being, I think for the end user it's actually less complicated.<BR>
They can select the package they're trying to find, the attributes of<BR>
the files they want, the short names of the files, or they can search<BR>
descriptions or other file or package meta-data. This query could be<BR>
in the form of a URI. A user wanting to find all logs in the system<BR>
could specify something like "<A HREF="file://attr:logs/apache2/access_log">file://attr:logs/apache2/access_log</A>", or<BR>
"<A HREF="file://pkg:apache2/logs/access_log">file://pkg:apache2/logs/access_log</A>" to get to the same file.<BR>
<BR>
I think the biggest win of this sort of system (given there aren't<BR>
problems I don't immediately see with it) is that every application<BR>
knows exactly where to find every library it needs, and it *knows*<BR>
that it will always be there. When one asks the question "Is XXX<BR>
installed on this system?" the answer will always be yes or no, and<BR>
never "Yes, but in /home/yyy" or "Yes, but I'm not sure where" or<BR>
"Yes, but its configuration files have been moved to zzz" or any other<BR>
of a million-kajillion things.<BR>
<BR>
/Complete/ uninstallation (and, hopefully, installation) of packages<BR>
with a single command is possible.<BR>
<BR>
A unified documentation system is possible. A file that contains the<BR>
index, TOC, and other meta-data of the documentation for a package can<BR>
be marked with an attr-documentation, and can be merged with the help<BR>
for all the packages on the system.<BR>
<BR>
Also, a common menu system is possible. Independent of a window manager<BR>
or a console program, there could be a menu of all applications,<BR>
configuration options, peripherals, and so on, on the system.<BR>
<BR>
Zero configuration required. That's the goal. You install something,<BR>
and you don't have to tell it where to find things, or create shortcuts<BR>
to it, or integrate it with your configuration system. All integration<BR>
is done automatically. You might have to tell it some stuff about how<BR>
to behave ("Which port should I listen on?") but that's part of the<BR>
installation process. Sensible, secure, usable defaults.<BR>
<BR>
The biggest concern with the OS dictating things like the<BR>
documentation system, the menu system, and the configuration system is<BR>
that while these things probably need to be unified and standardized,<BR>
this doesn't give a lot of room for improvement--it's a big commitment--<BR>
so you /really/ need to get it right the first time. It also needs to<BR>
be designed so that administrators still have complete control over<BR>
everything they /need/ to have control over, and no more. Unix<BR>
definitely goes the wrong direction with this, and Windows goes too far.<BR>
But I think the benefits of standardizing everything you can about a<BR>
system without removing too much flexibility are more than great<BR>
enough to justify it.<BR>
<BR>
While Microsoft's operating systems don't exactly get this done<BR>
elegantly, the fact that things can be installed on them with<BR>
/no trouble/ and /no hassle/ is a result of the fact that applications<BR>
know where to find things. There's an API (or a registry location, I'm<BR>
not sure) that tells you where the Program Files directory is, where<BR>
the My Documents directory is, where the Java Runtime is (either Sun's<BR>
or MS's, it doesn't matter--it gets it right). Shared libraries always<BR>
have the same name on every machine, and are always located in the<BR>
same place, %windir%\system32. Most of the ideas for SOS, however,<BR>
came from MS's .NET assembly versioning and the GAC.<BR>
<BR>
I wholeheartedly believe that SOS's filesystem lacks only the<BR>
important property of automatic, transparent serialization<BR>
deserialization of Lisp data structures before it's a true POS. And<BR>
this could be incrementally added later. It fits right in with the<BR>
packages-as-basis-of-organization idea, too. The difference, to a<BR>
package, of "file" vs. "data structure" doesn't /need/ to be<BR>
clear-cut. It's all part of the same package, right?<BR>
<BR>
One more thought. Instead of user-based security, resources could be<BR>
secured from packages. The web server package would have access to<BR>
network resources and web content, the user's package would have<BR>
access to his or her documents and user programs, and most packages would<BR>
have no access to files in other packages except for dependencies.<BR>
<BR>
> [...] Also, Movitz is still so immature that any substantial project<BR>
> using it would be likely to influence Movitz as a platform also.<BR>
<BR>
I'm afraid that, more than the security question, the question of<BR>
filesystems and package systems are irrelevant to Movitz proper, but<BR>
you may know better than me about the implementation requirements, Frode.<BR>
<BR>
<BR>
Chris Capel<BR>
</FONT>
</P>
</BODY>
</HTML>