Parse project .info files: present module list and dependency information

FYI: you can already browse projects by category (which is now the default at http://drupal.org/project/Modules, in fact). when you do, there's a whole category for CCK and related modules: http://drupal.org/project/Modules/category/88. similarly, there are categories for Views (http://drupal.org/project/Modules/category/89) and Organic Groups (http://drupal.org/project/Modules/category/90) 3 of the biggest sets of modules that depend on each other in the way you describe...

also, many projects that currently depend on others are fairly good about documenting this and adding links in the project descriptions themselves.

however, it's true that a more formalized dependency system would be nice, if nothing else to standardize this info and visually format it consistantly. we've already got this in the .info files in 5.x core, in fact, so it makes some sense to do that to project nodes, too.

however, i'd be slightly worried about people not keeping these 2 sources of information in sync (and having the project.module parse the .info files and use them directly seems a little scary). also, in the future, you'll be able to specific version-specific dependencies (e.g. og-5.x-1.0 depends on views 5.x-1.0 or later, but og-5.x-1.2 requires views 5.x-1.1 (or something)). to continue that example, og 4.7.x-* does not depend on views at all, but 5.x-* does. it's not obvious if/how we can represent that via project-level dependencies as you're suggesting.

i'm not marking as "won't fix", i'm just sharing my ideas (and concerns), too. clearly, this needs more thought if anything's going to come of it.

thanks,
-derek

I did expect that there would be a few concerns (and many of your concerns were thoughts of mine as well). I think that parsing the .info file would be an option, and a lot less difficult than having the maintainers keep up with it seperately.

I think that having the ability to mark a module as a "library" module would be a good course of action though, because it doesn't make sense to list a module on the Modules download page if the module doesn't really do anything... IMO anyways. (Maybe there should be a way to specify a "library" or API module in the .info file). One thing to note, though, is that some modules provide both functionality, and a library for other modules to use... maybe once there is a way to specify library modules, developers should be encouraged to split off library functionality other functionality (or maybe there should be a "library" project type on drupal.org).

I realize that there are categories right now, but even still, there are a lot of projects to wade through. I'd like for things to be a little less cluttered, but I realize that it's going to add a level of complexity, so I'm not really sure if it's worth it or not.

I think all of these would be useful to have. Dependancies can be taken from the .info file.

well, one of the main reasons I like out .ini format is that the info can there can be used safely by PHP code like project module. i think project should reuse most of the info in the .ini, to be honest. some disagree, which is fine. dependencies are a good place to start. not sure what the UI should look like. maybe just state the dependencies on the project page for now.

if we put code on d.o to parse .info files from drupal modules, it needs to go into project_d_o_hacks.module, not project.module, or else we undo a year's worth of effort on my part to try to make project.module useful for other sites.

the potential for supporting project dependencies and relationships in project.module itself is a reasonable (if low priority) feature reuqest. however, hard-coding .info file logic is out of the question. we need a separate module for that.

See also: http://groups.drupal.org/node/11998

This would be a useful thing to have.

Since dependencies are at the module rather than the project level, I think what we need is actually a project_info module that parses and stores from .info files:

* the association of modules and their titles and descriptions with projects (projects being identified by nid), tracked by version, and
* the dependencies of modules on each other, again tracked by version

From this, we could present e.g. in blocks that appear on a project's page:

1. The projects that this project depends on.

2. The projects that depend on this project.

3. The modules included in this project (with their titles and descriptions).

I'll have a go at sketching this in as a new module.

I'm changing the issue title to reflect this specific aim. A "library" categorization of modules looks to me like a separate feature request (possibly just a new vocabulary or term).

Versioning information is going to be tricky.

In the first place, dependencies vary between core compatibility versions. module_x's Drupal 5 version will have different dependencies than its Drupal 6 version. So we can't say module_x has an overall set of dependencies. Rather, we need to distinguish between dependencies by version.

But dependencies may also vary within a core compatibility version. E.g., a core 6.x version of a given module may exist in multiple releases. Each of these may have different dependencies.

And we know version information only on one side of this dependency. That is, we know that the 6.2 stable release of module_x is dependent on module_y, but we don't know what version of module_y is required (6.1, 6.2, etc.).

Possibly we need to track dependencies only by Drupal core compatibility--e.g., assign module_x a single set of dependencies for Drupal 6.

It's not going to be straightforward deciding how to display this information. But first things first--we need to capture it before displaying.

@nedjo: It's great you're going to be working on this. If you haven't yet, please read http://groups.drupal.org/node/11998#comment-39642. Thanks!

Here's a first very rough draft, completely untested.

Some quick notes:

* Creates a new module, project_info, which is in an info directory in project.

* We capture information on modules and their dependencies by calling a processing function from the packaging script.

* Modules are stored in a new table, project_info, with their associated release node ID and the various data in the .info file (name, description, etc.). Looking again at this table, it may be too normalized--it may be better to store e.g. the core compatibility tid and the project node nid here too, to reduce the need to join on other tables.

* Modules themselves have a release associated with them, but for dependencies all we have is a module name. So we store dependencies as a relationship between a module ID (the ID field of the project_info table) and the dependency's module name.

A lot still to do. I'll pick this up later in the week. Comments meantime or suggestions for how I could test would be great.

subscribing, i guess we probably want to bump this forward to 6.x right?

Subscribe

I've been working on this stuff recently, but from the perspective of an outside system.

http://groups.drupal.org/node/21295

Basically I am building a directory structure of easily mirror-able meta-information, that can be parsed on the client side.
This is specifically with the goal of writing something like apt-get for Drupal.

I'm already working on automating these dependencies in my code, but I'd very happily support this getting into the generated info files itself (I already use the values from the .info files where available)

Something to keep in mind, is that modules provided by a package might change between releases, so this shouldn't be tied to the project, but to the releases (I know it makes it more complex, but that's how it works)

Same case for dependencies. New dependencies might be added or removed.

I'm going to be working on this since I need it for project_issue_file_test (PIFT).

Initial patch to confirm direction.

This should store the list of modules.

This should store dependencies to the highest dev release in the same drupal core API version...later we can add support for the advanced style .info files supported by D7.

I'll add some sort of views support for this.

Module list view done, got to head out...I'll get it and dependency view done before posted.

With views!

Added API.

Now I will attempt to integrate this with pifr.

PIFT integration was simple:

I'll work on deploying on d6.d.o and testing the .info file parsing on release build, and I suppose I need to figure out some sort of script to run on all projects as we talked about.

The view will need to be tailored to a particular release for a project since the current argument is just pid, but that is minor since we don't need to display them...we really need the parsing for pift.

This patch is update to a new schema that keeps track of the specific "module" that is depended on in addition to the release.

After quick chat with dww:

Plan: copy package-release-nodes.php as parse-legacy-info.php and hack it to run in tmp and not touch db.

Preferably come up with hack to temporarily store dependencies and parse in second round without having to re-checkout everything.

More up-to-date, still needs work to finish #24.

Seems to work well, I'll wait for d6.d.o to come back up for final testing.

Cleaned version.

Through testing on d6.d.o fixed this so far, plus requires #424372: :: in .info files causes Fatal error.

Seems to be working well.

Latest patch against PIFT.

Ass patch file.