Messages in this thread | | | From | Dan Williams <> | Date | Mon, 29 Mar 2021 14:03:37 -0700 | Subject | Re: [PATCH 2/4] cxl/mem: Fix cdev_device_add() error handling |
| |
On Thu, Mar 25, 2021 at 10:12 AM Jason Gunthorpe <jgg@nvidia.com> wrote: > > On Wed, Mar 24, 2021 at 02:01:56PM -0700, Dan Williams wrote: > > If cdev_device_add() fails then the allocation performed by > > dev_set_name() is leaked. Use put_device(), not open coded release, for > > device_add() failures. > > > > The comment is obsolete because direct err_id failures need not worry > > about the device being live. > > > > The release method expects the percpu_ref is already dead, so > > percpu_ref_kill() is needed before put_device(). However, given that the > > cdev was partially live wait_for_completion() also belongs in the > > release method. > > > > Fixes: b39cb1052a5c ("cxl/mem: Register CXL memX devices") > > Reported-by: Jason Gunthorpe <jgg@nvidia.com> > > Signed-off-by: Dan Williams <dan.j.williams@intel.com> > > drivers/cxl/mem.c | 16 ++++++---------- > > 1 file changed, 6 insertions(+), 10 deletions(-) > > > > diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c > > index 30bf4f0f3c17..e53d573ae4ab 100644 > > +++ b/drivers/cxl/mem.c > > @@ -1049,6 +1049,7 @@ static void cxl_memdev_release(struct device *dev) > > { > > struct cxl_memdev *cxlmd = to_cxl_memdev(dev); > > > > + wait_for_completion(&cxlmd->ops_dead); > > This only works because the fops stuff is not right, a kref shouldn't > have a completion like this. > > Also, don't use devm for unregister. That just makes it extra-hard to > write the driver remove function correctly.
To date there is no driver remove function, however if that changes then I expect all the devm needs to go.
> > > @@ -1157,7 +1158,6 @@ static void cxlmdev_unregister(void *_cxlmd) > > > > percpu_ref_kill(&cxlmd->ops_active); > > cdev_device_del(&cxlmd->cdev, dev); > > - wait_for_completion(&cxlmd->ops_dead); > > cxlmd->cxlm = NULL; > > put_device(dev); > > } > > @@ -1210,20 +1210,16 @@ static int cxl_mem_add_memdev(struct cxl_mem *cxlm) > > cdev_init(cdev, &cxl_memdev_fops); > > > > rc = cdev_device_add(cdev, dev); > > - if (rc) > > - goto err_add; > > + if (rc) { > > + percpu_ref_kill(&cxlmd->ops_active); > > + put_device(dev); > > This must be one high performance ioctl to warrant the percpu ref.. If > it is not high performance use a rwsem, otherwise I'd suggest srcu as > a faster/simpler alternative.
The plan is to refactor and share the same reference counted fops mechanism as debugfs and make that common infrastructure. However, in the meantime I think global srcu is suitable.
> > This is a use-after-free: > > static long cxl_memdev_ioctl(struct file *file, unsigned int cmd, > unsigned long arg) > { > struct cxl_memdev *cxlmd; > struct inode *inode; > int rc = -ENOTTY; > > inode = file_inode(file); > cxlmd = container_of(inode->i_cdev, typeof(*cxlmd), cdev); > ^^^^^ can be freed memory > > ioctl needs to store the cxlmd in file->private_data and > open()/release() need to do get/put device on it so the memory stays > around. This is why open gets the inode as an argument and ioctl/etc > does not.
Ugh, exactly why I was motivated to attempt to preclude this with new core infrastructure that attempted to fix this centrally [1]. Remove the possibility of "others" getting this wrong. However after my initial idea bounced off Greg then I ended up shipping this bug in the local rewrite. I think the debugfs api gets this right in terms of centralizing the reference count management, and I want to see something similar for common driver ioctl patterns.
[1]: http://lore.kernel.org/r/CAPcyv4hGxLZGEkfnqdLfF-a1CzfEjLux-TBxXztbknFhEe9mYA@mail.gmail.com
> > The ordering cxlmdev_unregister should mirror the ordering in create > so cdev_device_del should be first
Sure.
| |