请教CFS调度器问题2

donghaitad

static struct sched_entity *pick_next_entity(struct cfs_rq *cfs_rq)
{
        struct sched_entity *se = NULL;

        if (first_fair(cfs_rq)) {
                se = __pick_next_entity(cfs_rq);
                printk("XXXXXXXXX!\n");
                set_next_entity(cfs_rq, se);
        }

        return se;
}

本人仅加了一个printk()函数，单CPU环境下系统就卡住了，但多CPU环境下系统却可以正常运行。

想请大家指教!非常感谢！

humjb_1983

donghaitad 发表于 2013-10-31 15:51
static struct sched_entity *pick_next_entity(struct cfs_rq *cfs_rq)
{
        struct sched_entity *se = N ...

这个函数的调用频度非常高，printk太频繁是会将内核搞死的，具体原因忘了，redhat的bugzilla中应该提过这个问题，可以google一下。

embeddedlwp

要不用printk_ratelimit试试。

firkraag

Maybe deadlock:

printk can call wake_up to wake up the klogd daemon. This will deadlock on aquiring the scheduler lock of the local run queue.

donghaitad

非常感谢，我找了个补丁问题解决了。

Re: [PATCH] printk deadlocks if called with runqueue lock held

From: Steven Rostedt
Date: Fri Jan 18 2008 - 06:36:24 EST
Next message: Miklos Szeredi: "Re: [PATCH -v6 2/2] Updating ctime and mtime for memory-mappedfiles"
Previous message: Jarek Poplawski: "Re: [PATCH 7/7] driver-core : convert semaphore to mutex in structclass"
In reply to: Jiri Kosina: "Re: [PATCH] printk deadlocks if called with runqueue lock held"
Next in thread: Jan Kiszka: "Re: [PATCH] printk deadlocks if called with runqueue lock held"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

On Fri, 18 Jan 2008, Jiri Kosina wrote:
>
> If this patch is going to be merged, you should perhaps adjust the comment
> introduced by the above mentioned commit, so that it reflects the new
> behavior.

Thanks for pointing this out. Updated patch below:

-- Steve
=========

I thought that one could place a printk anywhere without worrying.
But it seems that it is not wise to place a printk where the runqueue
lock is held.

I just spent two hours debugging why some of my code was locking up,
to find that the lockup was caused by some debugging printk's that
I had in the scheduler. The printk's were only in rare paths so
they shouldn't be too much of a problem, but after I hit the printk
the system locked up.

Thinking that it was locking up on my code I went looking down the
wrong path. I finally found (after examining an NMI dump) that
the lockup happened because printk was trying to wakeup the klogd
daemon, which caused a deadlock when the try_to_wakeup code tries
to grab the runqueue lock.

Since printks are seldom called with interrupts disabled, we can
hold off the waking of klogd if they are. We don't have access to
the runqueue locks from printk, but those locks need interrupts
disabled in order to be held.

Calling printk with interrupts disabled should only be done for
emergencies and debugging anyway.

And with this patch, my code ran fine

Signed-off-by: Steven Rostedt <srostedt@xxxxxxxxxx>
---
kernel/printk.c | 16 ++++++++++++----
1 file changed, 12 insertions(+), 4 deletions(-)

Index: linux-mcount.git/kernel/printk.c
===================================================================
--- linux-mcount.git.orig/kernel/printk.c        2008-01-18 06:29:15.000000000 -0500
+++ linux-mcount.git/kernel/printk.c        2008-01-18 06:32:38.000000000 -0500
@@ -595,9 +595,11 @@ static int have_callable_console(void)
* @fmt: format string
*
* This is printk(). It can be called from any context. We want it to work.
- * Be aware of the fact that if oops_in_progress is not set, we might try to
- * wake klogd up which could deadlock on runqueue lock if printk() is called
- * from scheduler code.
+ *
+ * Note: if printk() is called with interrupts disabled, it will not wake
+ * up the klogd. This is to avoid a deadlock from calling printk() in schedule
+ * with the runqueue lock held and having the wake_up grab the runqueue lock
+ * as well.
*
* We try to grab the console_sem. If we succeed, it's easy - we log the output and
* call the console drivers. If we fail to get the semaphore we place the output
@@ -978,7 +980,13 @@ void release_console_sem(void)
console_locked = 0;
up(&console_sem);
spin_unlock_irqrestore(&logbuf_lock, flags);
-        if (wake_klogd)
+        /*
+         * If we try to wake up klogd while printing with the runqueue lock
+         * held, this will deadlock. We don't have access to the runqueue
+         * lock from here, but just checking for interrupts disabled
+         * should be enough.
+         */
+        if (!irqs_disabled() && wake_klogd)
wake_up_klogd();
}
EXPORT_SYMBOL(release_console_sem);
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/