Problem statement
If IBM Spectrum Scale is configured with AD RFC2307 file authentication and you face either of the following issues:
1) NFS protocol access issue with some AD user (e.g. NFS permission denied)
2) SMB protocol access issue with AD user (NT_STATUS_LOGON_FAILURE)
3) failure in resolving some AD user on IBM Spectrum Scale protocol node
then one of the reason could be WBC_ERR_DOMAIN_NOT_FOUND / NT_STATUS_NONE_MAPPED error under the hood.
This blog explains the possible solution for this error. It leverages capability of mmadquery utility in IBM Spectrum Scale to determine the failure cause.
Failure Examples
1. file protocol access issue (e.g. NFS permission denied) with some AD user
# WBC_ERR_DOMAIN_NOT_FOUND in NFS server log for permission denied issue
2019-03-26 15:00:09 : epoch 0003004b : rh73n2 : gpfs.ganesha.nfsd-15043[work-101] nfs_req_creds :ID MAPPER :WARN :Could not map principal user1@XYZ.COM to uid
2019-03-26 15:00:09 : epoch 0003004b : rh73n2 : gpfs.ganesha.nfsd-15043[work-102] principal2uid :ID MAPPER :CRIT :wbcSidToUid for uid returned WBC_ERR_DOMAIN_NOT_FOUND
2. SMB protocol access issue with AD user (NT_STATUS_LOGON_FAILURE)
# NT_STATUS_NONE_MAPPED error reported in winbind.log for SMB access
$ smbclient //rh73n/smbexp_fset1 -m smb3 -U GANESHA\aduser2%Passw0rd -c "ls"
session setup failed: NT_STATUS_LOGON_FAILURE
3. Failure in resolving AD user
# WBC_ERR_DOMAIN_NOT_FOUND while resolving AD user on IBM Spectrum Scale
$ /usr/lpp/mmfs/bin/wbinfo -i GANESHA\\aduser1
failed to call wbcGetpwnam: WBC_ERR_DOMAIN_NOT_FOUND
Could not get info for user GANESHA\aduser1
Solution for above failures
Following will help to solve above failures. Ensure:
– uidNumber and primaryGroupID set for that user
– uidNumber and primaryGroupID values must fall in –unixmap-domains range
An example to explicate the problem and suggested solution.
1. IBM Spectrum Scale configured with AD RFC2307 file authentication with “GANESHA” domain and uid/gid range “9000-30000”.
$ mmuserauth service create --data-access-method file --type ad --servers 192.168.122.151 --user-name Administrator --pwd-file adprivate.pwdfile --netbios-name rh73n --idmap-role master --unixmap-domains "GANESHA(9000-30000)"
File authentication configuration completed successfully.
2. Test Setup of two users from AD:
I’ve defined two users in AD as below:
aduser1 – uidNumber and primaryGroupID not set
aduser2 – primaryGroupID not set
(For reference, see image below for user aduser1 uidNumber and primaryGroupID.
– primaryGroupID not set because “Domain Users” GID doesn’t fall within –unixmap-domains range “9000-30000” configured on the cluster
– uidNumber is not set)
3. Verification using mmadquery command for problematic users:
You can also check above settings on IBM Spectrum Scale using mmadquery command. Field “uidNumber” displays uidNumber and field “primaryGroupGid” displays primaryGroupID.
In the following output, you can validate that uidNumber and primaryGroupID not set for aduser1, primaryGroupID not set for aduser2. This will lead to the problems mentioned in “Failure Examples” section above.
$ mmadquery list uids --server 192.168.122.151 --domain GANESHA.DEV.NET --user Administrator --pwd-file /tmp/pwdfile -L -Y mmadquery:uids_long:HEADER:version:reserved:reserved:user:sid:uid:uidNumber:gidNumber:primaryGroupId:primaryGroupGid:
mmadquery:uids_long:0:1:::aduser1:S-1-5-21-3530651294-1835310728-4022169620-1118::::1103::
mmadquery:uids_long:0:1:::aduser2:S-1-5-21-3530651294-1835310728-4022169620-1119::15002::1103::
4. Try to resolve AD users aduser1 and aduser2 – it fails (due to missing uidNumber and/or primaryGroupID values).
$ /usr/lpp/mmfs/bin/wbinfo -a GANESHA\\aduser1%Passw0rd
plaintext password authentication succeeded
challenge/response password authentication succeeded
$ /usr/lpp/mmfs/bin/wbinfo -i GANESHA\\aduser1
failed to call wbcGetpwnam: WBC_ERR_DOMAIN_NOT_FOUND
Could not get info for user GANESHA\aduser1
$ /usr/lpp/mmfs/bin/wbinfo -a GANESHA\\aduser2%Passw0rd
plaintext password authentication succeeded
challenge/response password authentication succeeded
$ /usr/lpp/mmfs/bin/wbinfo -i GANESHA\\aduser2
failed to call wbcGetpwnam: WBC_ERR_DOMAIN_NOT_FOUND
Could not get info for user GANESHA\aduser2
5. Set uidNumber and primaryGroupID for the users in Active Directory:
On AD server, Set uidNumber and primaryGroupID for aduser1.
Set primaryGroupID for aduser2. Make sure uidNumber and primaryGroupID values fall within –unixmap-domains range configured on cluster.
– uidNumber can be set through “Attribute Editor” -> uidNumber attribute.
– primaryGroupID can be set through “Members Of” -> Set primary group to Group that falls within –unixmap-domains range
(For reference, modified values for aduser1 are shown in following image
uidNumber=15001 and primaryGroupID=10001. Both these values are within –unixmap-domains range which is “9000-30000”.)
6. Verification using mmadquery command.
You can verify the modified uidNumber and primaryGroupID on IBM Spectrum scale using mmadquery command.
$ mmadquery list uids --server 192.168.122.151 --domain GANESHA.DEV.NET --user Administrator --pwd-file /tmp/pwdfile -L -Y mmadquery:uids_long:HEADER:version:reserved:reserved:user:sid:uid:uidNumber:gidNumber:primaryGroupId:primaryGroupGid: mmadquery:uids_long:0:1:::aduser1:S-1-5-21-3530651294-1835310728-4022169620-1118::15001::1103:10001:
mmadquery:uids_long:0:1:::aduser2:S-1-5-21-3530651294-1835310728-4022169620-1119::15002::1103:10001:
7. Resolving AD users succeed.
Try to resolve AD users again. They do succeed.
$ /usr/lpp/mmfs/bin/wbinfo -a GANESHA\\aduser1%Passw0rd
plaintext password authentication succeeded
challenge/response password authentication succeeded
$ /usr/lpp/mmfs/bin/wbinfo -i GANESHA\\aduser1
GANESHA\aduser1:*:15001:10001:aduser1:/home/GANESHA/aduser1:/bin/false
$ /usr/lpp/mmfs/bin/wbinfo -a GANESHA\\aduser2%Passw0rd
plaintext password authentication succeeded
challenge/response password authentication succeeded
$ /usr/lpp/mmfs/bin/wbinfo -i GANESHA\\aduser2
GANESHA\aduser2:*:15002:10001:aduser2:/home/GANESHA/aduser2:/bin/false
8. SMB access succeeds:
Try to perform SMB access with AD user aduser2 (for example 3 failure). It succeeds this time because primaryGroupID for aduser2 is now set.
$ smbclient //rh73n/smbexp_fset1 -m smb3 -U GANESHA\aduser2%Passw0rd -c "ls"
Domain=[GANESHA] OS=[] Server=[]
. D 0 Mon Jul 15 23:45:29 2019
.. D 0 Sat Jul 20 00:02:57 2019
file1 A 8 Mon Jul 15 23:45:29 2019
In above example, IBM Spectrum Scale was configured to fetch primary group as Windows primary group of a user on the Active Directory. It is identified by primaryGroupID attribute on Active Directory.
You can also configure IBM Spectrum Scale to fetch primary group as set in “UNIX attributes” of a user on the Active Directory e.g. –unixmap-domains “GANESHA(20000-50000:unix)”. In such case, you need to ensure gidNumber is set for AD user to avoid above failures (mmadquery displays it as gidNumber field).
“These are my personal views and do not necessarily reflect that of my employer” .
crk10.wordpress.com 