If you can break a program into several separate applications, then you can use SELinux to control what each application is allowed. Then SELinux could prevent a hacked application from doing more then expected.
The pam stack was invented a long time ago to allow customizations of the login process. One problem with the pam_stack is it allowed programmers to slowly hack it up to give the programs more and more access. I have seen pam modules that do some crazy stuff.
Since we confine login applications with SELinux, we sometimes come in conflict with some of the more powerful pam modules.
We in the SELinux world want to control what login programs can do. For example we want to stop login programs like sshd from reading/writing all content in your homedir.
Why is this important?
Over the years it has been shown that login programs have had bugs that led to information leakage without the users ever being able to login to a system.
One use case of pam, was the requirement of creating a homedir, the first time a user logs into a system. Usually colleges and universities use this for students logging into a shared service. But many companies use it also.
The pam_mkhomedir PAM module will create a users home directory if it does not exist when the session begins. This allows users to be present in central database (such as NIS, kerberos or LDAP) without using a distributed file system or pre-creating a large number of directories. The skeleton directory (usually /etc/skel/) is used to copy default files and also sets a umask for the creation.
This means with pam_mkhomedir, login programs have to be allowed to create/read/write all content in your homedir. This means we would have to allow sshd or xdm to read the content even if the user was not able to login, meaning a bug in one of these apps could allow content to be read or modified without the attacker ever logging into the machine.
The pam_oddjob_mkhomedir.so module checks if the user's home directory exists, and if it does not, it invokes the mkhomedirfor method of the com.redhat.oddjob_mkhomedir service for the PAM_USER if the module is running with superuser privileges. Otherwise, it invokes the mkmyhome‐dir method.
The location of the skeleton directory and the default umask are deter‐mined by the configuration for the corresponding service in oddjobd-mkhomedir.conf, so they can not be specified as arguments to this module.
If D-Bus has not been configured to allow the calling application to invoke these methods provided as part of the com.redhat.oddjob_mkhome‐dir interface of the / object provided by the com.redhat.oddjob_mkhome‐dir service, then oddjobd will not receive the request and an error will be returned by D-Bus.
Nalin Dahyabhai wrote pam_oddjob_mkhomedir many years ago to separate out the ability to create a home directory and all of the content from the login programs. Basically the pam module sends a dbus signal to a dbus service oddjob, which launches a tool to create the homedir and its content. SELinux policy is written to allow this application to succeed. We end up with much less access required for the login programs.
If you want the home directory created at login time if it does not exist. Use pam_oddjob_mkhomedir instead of pam_mkhomedir.