Why would portald not update blocks?
I'm working with Slash 2.2.5 for the past few days, the install went fine as far as I can see on this RedHat 7.3 box. The only problem behavior I'm having so far is getting blocks with RDF content to work, any of them.
The way I understand it, portald fetches the RDF specified in the block definition and uses it to create static content, which it inserts into the block and the HTML for which can be seen in the admin block panel. Is this correct?
None of the blocks on my system have updated content, and in fact most of them have EMPTY content, which implies to me that they haven't been fetched by portald. The ones that do have content are way out of date or not real, and I believe were part of the initial block creation during installation.
My slashd.log implies that portald gets called and has no errors. If I directly access the RDF links by hand they are valid and accessible. (http://www.slashdot.org/apache.rdf for example.)
I have RTFM several times but am either missing something or I am just not understanding. I find little to no information about creating blocks. But so far I'm just trying to get the blocks to work that came with the package.
My slashd.log says stuff like this:
Tue Jun 4 19:24:21 2002 processed /usr/local/slash/site/slash.pumpsoft.net / asks; 19 successful (adminmail.pl daily.pl daily_archive.pl daily_forget.pl expire.pl flush_formkeys.pl freshenup.pl message_delete.pl message_delivery.pl new_headfoot.pl new_motd.pl open_backend.pl p2f_cheesy.pl p2f_hof_topics.pl refresh_authors_cache.pl run_moderatord.pl run_portald.pl set_recent_topics.pl spamarmor.pl); 0 failed
Tue Jun 4 19:24:21 2002 first task will be freshenup.pl in 0 secs
Tue Jun 4 19:24:21 2002 freshenup.pl begin
Tue Jun 4 19:24:21 2002 Updating 00/01/25/1236215
Tue Jun 4 19:24:22 2002 article.pl virtual_user=slash ssi=yes sid='00/01/25/1236215' section='articles' bytes=9959
Tue Jun 4 19:24:22 2002 freshenup.pl updated articles:00/01/25/1236215 (Now What?)
Tue Jun 4 19:24:22 2002 Updating 02/06/03/0348216
Tue Jun 4 19:24:23 2002 article.pl virtual_user=slash ssi=yes sid='02/06/03/0348216' section='articles' bytes=10043
Tue Jun 4 19:24:23 2002 freshenup.pl updated articles:02/06/03/0348216 (Test Story)
Tue Jun 4 19:24:23 2002 Updating 02/06/03/0740251
What have I missed? How do I get the blocks to update from the RDF's?
--
gasp
The way I understand it, portald fetches the RDF specified in the block definition and uses it to create static content, which it inserts into the block and the HTML for which can be seen in the admin block panel. Is this correct?
None of the blocks on my system have updated content, and in fact most of them have EMPTY content, which implies to me that they haven't been fetched by portald. The ones that do have content are way out of date or not real, and I believe were part of the initial block creation during installation.
My slashd.log implies that portald gets called and has no errors. If I directly access the RDF links by hand they are valid and accessible. (http://www.slashdot.org/apache.rdf for example.)
I have RTFM several times but am either missing something or I am just not understanding. I find little to no information about creating blocks. But so far I'm just trying to get the blocks to work that came with the package.
My slashd.log says stuff like this:
Tue Jun 4 19:24:21 2002 processed
Tue Jun 4 19:24:21 2002 first task will be freshenup.pl in 0 secs
Tue Jun 4 19:24:21 2002 freshenup.pl begin
Tue Jun 4 19:24:21 2002 Updating 00/01/25/1236215
Tue Jun 4 19:24:22 2002 article.pl virtual_user=slash ssi=yes sid='00/01/25/1236215' section='articles' bytes=9959
Tue Jun 4 19:24:22 2002 freshenup.pl updated articles:00/01/25/1236215 (Now What?)
Tue Jun 4 19:24:22 2002 Updating 02/06/03/0348216
Tue Jun 4 19:24:23 2002 article.pl virtual_user=slash ssi=yes sid='02/06/03/0348216' section='articles' bytes=10043
Tue Jun 4 19:24:23 2002 freshenup.pl updated articles:02/06/03/0348216 (Test Story)
Tue Jun 4 19:24:23 2002 Updating 02/06/03/0740251
What have I missed? How do I get the blocks to update from the RDF's?
--
gasp
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
For one thing (Score:2)
See slash bug regarding this [sourceforge.net]
lottadot [lottadot.com]
Re:For one thing (Score:0)
I am having no apparent problems with index.shtml being generated, or with posting stories or comments, and slashboxes are drawn on the index page. It's just that every slashbox that uses an RDF for content has never been updated and has only the original static HTML that came with the install files.
Even if this _was_ a problem with refreshing index.shtml, I would expect the correct contents in the Blocks panel for portald blocks that use RDFs. Since all the blocks have empty or stale content in the panel, it looks like portald is never downloading the RDF links and updating the block content, although there are no errors in the log.
As I said, I've verified that the RDF links are valid and that portald seems to be getting called with no errors. It's just that it seems to have no visible effect, yet as far as I can tell the other parts of the site are working fine.
Also, I wasn't able to login here, so posted this as AC. (gasp)
Restart Apache (Score:1)
Re:Restart Apache (Score:0)
For example, I can't view the Apache block because it has no content. Blocks with completely empty content don't display. The "block" field in the block panel for the apache block is empty. IIRC, the RDF path is set to http://www.slashdot.org/apache.rdf, and I can manually wget that URL just fine. Yet that portald slashbox has _never_ had it's content updated from that RDF and has remained empty.
Another example, the secfocus block. It also uses and RDF path as a source, but the install set for that block included "dirty" static content, presumably from a previous RDF fetch by the build system of the packager. All the content in my secfocus block are way stale and invalid, the links are no longer on the Security Focus site, again demonstrating that my system has never fetched that RDF and updated the block. It's the same for all my blocks.
Everything looks fine, I just need a magic bullet to cause these RDF files to be fetched periodically and used to update the block content. I believe that's what portald does, and that it's called via run_portald which is kicked off from slashd. All this seems to be happenning without error, so I don't know why portald isn't fetching and updating. I've spent a fair amount of time looking on the web and at my config for the cause. I guess I'll have to dig more into the code. I must have something wierd going on if I'm the only one to have experienced this. I'm more of a PHP guy for my own code, but I guess I'll have to dive in if nobody here has seen this problem.
Thanks!
gasp
dns (Score:2)
what does portald.log say? (Score:1)
Re:what does portald.log say? (Score:0)
I'll have to find out why the slashd.log says all these are being run but apparently aren't. Any quick tips for where to look first?
Thanks, I didn't know that portald or components other than slashd were supposed to make logs. This gives me a better idea of what's (not) happening.
gasp
Re:what does portald.log say? (Score:1)
yeah, lots of tasks create their own log files (portald, messaged, moderatord) ... check your log file and make sure it's writable by nobody (or your equivilent) .. allthough, i can't imagine how your slashd.log would be updated if it isn't.
I didn't notice it in your snippet of your slashd.log, but you should be seeing something like this...
if not, that's your fisrst problemRe:what does portald.log say? (Score:0)
All the directories are owned and writeable by the user nobody, I see no problems there.
The slashd.log I posted wasn't really a snippet, it was the whole log. And there are _no_ entries like the ones you mention. It looks like my slashd isn't running a lot of things, I wonder why. As I'm on RedHat, to fire up slashd I'm using
Here's my entire slash.log file. I've restarted it a couple times or so in the past couple days as you can see. I assume there are a lot of things missing that I didn't know about:
#cat slashd.log
Tue Jun 4 19:24:21 2002 Starting slashd with pid 18394
Tue Jun 4 19:24:21 2002 Starting up Slashd (verbosity 2) with pid 18394
Tue Jun 4 19:24:21 2002 processed
Tue Jun 4 19:24:21 2002 first task will be freshenup.pl in 0 secs
Tue Jun 4 19:24:21 2002 freshenup.pl begin
Tue Jun 4 19:24:21 2002 Updating 00/01/25/1236215
Tue Jun 4 19:24:22 2002 article.pl virtual_user=slash ssi=yes sid='00/01/25/1236215' section='articles' bytes=9959
Tue Jun 4 19:24:22 2002 freshenup.pl updated articles:00/01/25/1236215 (Now What?)
Tue Jun 4 19:24:22 2002 Updating 02/06/03/0348216
Tue Jun 4 19:24:23 2002 article.pl virtual_user=slash ssi=yes sid='02/06/03/0348216' section='articles' bytes=10043
Tue Jun 4 19:24:23 2002 freshenup.pl updated articles:02/06/03/0348216 (Test Story)
Tue Jun 4 19:24:23 2002 Updating 02/06/03/0740251
Wed Jun 5 21:01:43 2002 Exiting slashd (TERM) with pid 18394
Wed Jun 5 21:01:47 2002 Starting slashd with pid 20469
Wed Jun 5 21:01:47 2002 Starting up Slashd (verbosity 2) with pid 20469
Wed Jun 5 21:01:47 2002 processed
Wed Jun 5 21:01:47 2002 first task will be freshenup.pl in 0 secs
Wed Jun 5 21:01:47 2002 freshenup.pl begin
Wed Jun 5 21:01:47 2002 Updating 02/06/03/0740251
Wed Jun 5 21:01:48 2002 article.pl virtual_user=slash ssi=yes sid='02/06/03/0740251' section='articles' bytes=12706
Wed Jun 5 21:01:48 2002 freshenup.pl updated articles:02/06/03/0740251 (test story Monday?)
Wed Jun 5 21:01:48 2002 Updating 02/06/03/1731206
Thu Jun 6 04:01:37 2002 Exiting slashd (TERM) with pid 20469
Thu Jun 6 04:01:40 2002 Starting slashd with pid 21078
Thu Jun 6 04:01:40 2002 Starting up Slashd (verbosity 2) with pid 21078
Thu Jun 6 04:01:40 2002 processed
Thu Jun 6 04:01:40 2002 first task will be freshenup.pl in 0 secs
Thu Jun 6 04:01:40 2002 freshenup.pl begin
Thu Jun 6 04:01:40 2002 Updating 00/01/25/1430236
Thu Jun 6 04:01:41 2002 article.pl virtual_user=slash ssi=yes sid='00/01/25/1430236' section='articles' bytes=15410
Thu Jun 6 04:01:41 2002 freshenup.pl updated articles:00/01/25/1430236 (You've Installed Slash!)
Thu Jun 6 04:01:41 2002 Updating 00/01/25/1236215
the problem is not portald (Score:1)
Everytime you restart slash, it is (correctly) loading all of the tasks, and starting "freshenup.pl" (the first task) .. but freshenup doesn't seem to ever be finishing. I have no idea why -- but freshenup is definitely your problem.
This is what a chunk of your slashd.log should look like when it freshenup notices a new story...
(all tasks that i know of have a begin/end line arround them ... and the end line allways telsl you what the next task it plans to run is, and when)
Re:the problem is not portald (Score:0)
But now it's taken another wierd turn. Now the slashd.log shows that freshenup.pl is completing. It's behaving better, although of the few tasks it's run (message_delivery.pl, set_recent_topics.pl) it still hasen't run portald yet. But I'll keep watching. Is there a way to change the priority or frequency that slashd calls these scripts? Also I don't have any other log files yet, only slashd.log.
But I'm really concerned here about not knowing the cause of the problem. I didn't change anything, I just added 6 lines to add extra calls to slashdLog() in freshenup.pl to help me find the stoppage. I restarted slashd. That's all. Now it seems to be working. Doesn't give me a lot of confidence in the stability of this thing.
Re:the problem is not portald (Score:1)
you should keep an eye on your logs for a couple of days. look for things that start and don't finish.
as for your other questions: tasks run based on the timespec in their run_ file ... look in your tasks directory to see what i mean. (they are specified in psuedo crontab format .. if anybody reading this doesn't kow what that means, search the www)
by default run_portald is set to only run once an hour (on my machine it's 37 minutes after every hour) ... others tasks are set to run at various times/frequencies.
like i said. keep an eye on your logs for about 72 hours ... do lots of stuff, post articles, post comments, post journals, create some flat discussions, moderate (as an admin since no one will have any points yet), metamoderate, and watch all the logs to see that everything says it's starting and stoping at least once a day (nothing runs less frequently then that)
Well, I found where it's hanging (Score:0)
my $w = $slashdb->getVar('writestatus', 'value');
if ($updates{$constants->{defaultsection}} ne "" || $w ne "ok") {
$slashdb->setVar("writestatus", "ok");
# it gets this far
prog2file(
"$constants->{basedir}/index.pl",
"$vu ssi=yes",
"$constants->{basedir}/index.shtml",
verbosity()
);
# but it never gets here. It dies in prog2file.
}
...
It hangs in prog2file() and never returns, but ONLY after I post a new story. If I restart slashd it doesn't hang, it goes all the way through until I post another story. So it only hangs if there is a new story since the last freshenup.
Since it took maybe 15 minutes after starting slashd before portald got called, I'm guessing that when I first set up slash, I posted a new story before portald got called, which hung freshenup and resulted in no other tasks getting called. It probably always hung after restarting slashd because there was always a new story waiting.
This time with no stories and being patient portald finally ran, but it only updated a few blocks, not all of them.
#cat portald.log
Thu Jun 6 05:37:02 2002 Starting portald with pid 22031
Thu Jun 6 05:37:02 2002 Launching Portald
Thu Jun 6 05:37:02 2002 Updating Portal Box Thingees
Thu Jun 6 05:37:03 2002 updated freshmeat
Thu Jun 6 05:37:04 2002 updated perlnews
Thu Jun 6 05:37:04 2002 updated slashcode
Thu Jun 6 05:37:05 2002 updated slashdot
Thu Jun 6 05:37:05 2002 updated top10journals
Thu Jun 6 05:37:05 2002 updated top10comments
Thu Jun 6 05:37:05 2002 updated rand
Thu Jun 6 05:37:05 2002 updated poll
Thu Jun 6 05:37:06 2002 updated uptime
Thu Jun 6 05:37:06 2002 Sucessfully Saved Portals
Thu Jun 6 05:37:06 2002 Exiting portald (exit) with pid 22031
Why weren't the other blocks updated? There are a LOT more RDF blocks than that, such as the apache block. Does the order number need to be non-negative, or what? Any way, I'm closer to finding the problem. Any more ideas would be helpful!
As the case is now, I'm still stuck until I find why prog2file is hanging freshenup.pl and stopping slashd from processing any more tasks. Please tell me the code is really more stable than this, I'd like to keep reverse engineering to a minumum! Thanks again
Re:Well, I found where it's hanging (Score:0)
Re:Well, I found where it's hanging (Score:0)
I notice that most other place that call prog2file() send an additional argument, and that this often corresponds to when a return value is used. It's getting late here so I'll look at prog2file tomorrow.
I started with a fresh install set, and freshenup.pl has so far been unable to create an index.shtml file, which is where this is breaking. The freshenup.pl only tries to make a new index.shtml when there is a new story waiting, and it's that prog2file() call intended to make a index.shtml from index.pl that is breaking it, hanging, and preventing any other tasks from running.
I don't have any guesses yet, except that if I'm the only one who has seen this problem it's likely I have something wrong with my environment thats either breaking the code or turning up a bug. My CPAN says everything is ok, and tests/reinstalls seem to bear that out. My Apache and MySQL are vanilla RedHat 7.3 out of the box with no changes. I'm not aware of anything else that's broken with Slash or any other apps on the box, so this is still a puzzle.
Re:Well, I found where it's hanging (Score:1)
Re:Well, I found where it's hanging (Score:0)
I didn't find any problems with the code, but my perl skills are weak for this type of debugging. And contrary to comments I've seen, IMHO the slash code is very poorly documented from an architecture viewpoint.
After coming to the conclusion that I was unlikely to find a code fix on my own, I decided to try installing on a different box from scratch. I had the exact same problem on that one too. So I decided to recompile _EVERYTHING_ from scratch instead of using the RedHat perl and apache packages that I have been. (I didn't think this would be related, since I'm not having any problems with the website or database, just the standalone slashd freshenup.pl task.)
Anyway, after a completely new compile from tarballs of perl, mysql, apache, and slash I still have exactly the same problem. The slash install FAQ is pretty simple, there doesn't seem to be much I can do wrong. And I'm pretty experienced with building apache and mysql during the past couple years, too. Everything just works fine, except for the prog2file() call in freshenup.pl that hangs.
Sigh. I'm real close to giving up on slash.
gasp
Re:Well, I found where it's hanging (Score:0)
prog2file() is hanging at this line:
$data = `$exec`;
I believe the value of $exec at this time is "/var/www/slash/site/my.hidden.domain/index.pl virtual_user=slash ssi=yes" based on the argument values in the call in freshenup.pl
Now, I'm not savvy in perl... Does the statement "$data = `$exec`;" actually execute the contents of $exec? It looks like just a normal string assignment to me. But something is obviously happening here, since it never gets past this line.
Boiled down even further (Score:1)
The line that's hanging ($data = `$exec`;) is apparently not a unique problem. Google turned up a small smattering of similar descriptions. In essence, sometimes when a perl script is executed this way as a system call from within another perl script, it runs completely, but the process fails to exit. (and thereby prevents the parent script from continuing.) Manually killing the child process works, and the parent script then continues on it's merry way. I didn't find an
Even more boiled (Score:1)
Digging even deeper shows that when index.pl hangs it is on the slashDisplay line near the end of main().
The odd thing at first glance is that when I manually kill the index.pl process, index.pl continues running and exits normally. This means to me that I must be killing a nested child process created by the slashDisplay() call in index.pl and it's something in that call that's doing the blocking.
The adventure continues, I guess. I'll try to find whe
End of the trail? (Score:1)
It's near the very end of slashDisplay(), the line that outputs all the HTML that the surrent instance has built. It's a simple print command that reads:
print $out unless $opt->{Return};
This is the culprit, but I have no idea why. If I add a command right before that line to p
Rant: Trying too hard (Score:1)
My point is, I'm used to having thin support, dealing with code that isn't ready for prime time, and getting my hands dirty. And my efforts to use slash are not so different really, except for one thing: for some reason I expected there to be more of a community around this code. I know it's written by perl monks and I'm certainly not one, but the barrier of entry for slash is way higher than it should be or could be.
I'm NOT bashing anybody here, or criticizing the existing work or documentation, I'm only saying there isn't enough of it. I've found many things unclear, undocumented, or missing. And yes, I know I'm free to look elsewhere. But before I get taken for a troll, please remember that I CHOOSE to bang my head against this particular wall, because I believe that this project is worth the effort. It's not like I haven't looked at PHP project alternatives and others (and I'm fluent in PHP.) Flatly speaking, my impression of the slash community so far has been somewhat "take-it-or-leave-it" bordering on unwelcoming.
No doubt my take on this is colored by the problems that I (and apparently others) have running slash, the long hours of troubleshooting, and very little response of even acknowledgment of the issue.
What does quoting with a ` character do in perl? (Score:1)
$data = `$exec`;
It's not just assigning the contents of $exec to $data, is it? If that were the intention, I see no need for the quotes at all. Did I just miss this in the language reference?
I had assumed that this was some perl shorthand for executing the contents of $exec and assigning the resulting output to $data, but I haven't found any language reference to support that. (It's slow going to debug code in an unfamiliar language
Re:What does quoting with a ` character do in perl (Score:1)
I see that using the backticks does indeed execute the value of the scalar as a command and return the output. Yes, it looked obvious, but remember I'm not a perl programmer.
This does indicate that it's index.pl that is failing, but that's not a certainty. The same command succeeds manually. Perhaps there's a problem with perl that causes a glitch when running a system command? If so it seems to be a wierd one and intermittent. Whatever the cause, it's not returning from executing the command.
I sure could use a slash/perl guru to make a guess on this one.
Re:Well, what next? (Score:1)
A slip of the mind... Anyway, it really doesn't seem to be article.pl after further testing. I put some logging lines in article.pl and manually tested them to be sure they work.
When prog2file gets to the $data = `$exec`; line it looks like article.pl is never actually executed. It looks like it's actually perl that's hanging with a failed attempt to execute the command. That's my best assessment right now.
This shifts the problem away from a coding bug and into the environment configuration. I must have something wrong with my perl setup, but I'm not sure where/how to go about finding it. I have the same results on a stock redHat 7.3 install and with a manual compile of all these tools. (I did use CPAN to fetch and install all the perl modules for both setups though.)
Geeze this is tiring. Is perl really this much of a bear to get working, or am I just being obtuse? Heh, I don't expect an answer, I know I'm probably talking to myself.