Joseph Fielding Smith, in general conference, said Brigham Young has been misunderstood and he meant Adam had keys.
Joseph Fielding Smith, Conference Report (April 1965): 10, 11
President Young Unjustly Condemned
Now I would like to express another thought which is vital to us one and all. President Brigham Young has been unjustly condemned for a statement he made to the effect that Adam is our God and the only one with whom we have to do. President Young's statement has been unmercifully condemned, but what he said is a righteous principle and in full accord with the doctrines of the kingdom of God. It is the doctrine of primogeniture in the kingdom of God and a glorious principle when it is fully and clearly understood. . . . we learn by virtue of the law of primogeniture, that all who are saved in the kingdom of God will be subject to Adam, for by divine appointment he holds these keys under the direction of Jesus Christ. I might carry this law a little further. According to the gospel of Jesus Christ, Joseph Fielding Smith will be subject to his father, and his father to his father in the family of God; and so it will go back from the end to the beginning; and we will all be obedient to Adam, whom the Prophet Joseph Smith declared holds the keys of salvation for his posterity who are redeemed, but "under the counsel and direction of the Holy One," who is Jesus Christ, who stands at the head because he is the Redeemer of the world, who gave us, through his atonement, the resurrection and eternal life if we will only repent and keep his laws and commandments.