The Pickering Fellowship is much more than a scholarship with paid internships. You are actually signing a contract with the federal government which you are bound to fulfill. While this contract is one that I am very proud and happy to fulfill, it is nonetheless a contract with consequences for not complying.
The most important point that you are agreeing to is to be a Foreign Service Officer for at least three years immediately after you graduate from your two-year masters. It cannot be a three year masters and it cannot be postponed other than for a Fulbright (no 1 year volunteer/teaching fellowships or Peace Corp). You must obtain two internships (don't worry, Pickering staff make it as easy as possible) and you will also do some workshops while you are in DC. You are also not exempt from the general rule that the first post for FSOs is assigned; you don't get to choose where you go and will likely do consular work the first tour.
You then are contractually obligated to fulfill the necessary steps to become a Foreign Service Officer. This includes the security clearance, the worldwide health clearance, the Foreign Service written test (FSOT) and the oral exam (FSOA). You also need to make sure you that you graduate from the program without letting your GPA drop below a 3.2. There is some flexibility with the GPA like academic probation and the chance to explain. Also, you cannot work over 20 hours a week.
The most critical things, the security clearance, medical clearance, and the two exams are requirements that may be troublesome for some. Luckily, there are workarounds for the exams, sort of. There is an alternative to passing the FSOT or FSOA (I don't remember which). If you don't pass, you don't break the agreement as far as I know. You just don't get tenure so will have to leave the Foreign Service after your three years. This is because you have to get tenure or get out; this is FS policy.
The medical clearance and the security clearance are something to be wary of. If you don't pass either of these, you won't be able to be an FSO and thus would owe Pickering/US gov the $40,000 that went to your tuition, for how ever many years they paid for you. There is some leeway in the medical clearance in that if they find a medical issue, you probably have time to clear it up and I've heard of instances where you are assigned to US posts until you are able to go abroad.
There is little leeway if you don't pass the security clearance, though. If you think you won't pass the security clearance, I would actually advise you to not apply. (Heartbreaking, I know. </3) You won't find out if you pass the clearance until after they pay for your first year. There is an appeal process that can be used, but it will likely mess with the timeline of your contractual obligation to do an internship every summer (clearance is needed for the required internships). The timeline change probably won't forfeit the agreement itself, though. More importantly I don't know how successful the appeal process is, but if you don't get the security clearance, you will be in a huge debt to the government if they decide to make you pay it back. During the appeal, you will likely continue with the process of applying to graduate schools because not doing so will definitely forfeit the agreement. While waiting for the results of the appeal mean, you may have to pay for your first year of graduate studies yourself (if you are an undergraduate fellow).
It is a bit scary to have so much on the line. Not passing the security clearance will put you in a tough position but to put things back into perspective, not passing is probably pretty unlikely. The biggest reasons, as far as I've seen, are unmanaged debt that you do not take care, a history of crime, and being at risk of foreign influence. Since most of those applying are young and academically minded (i.e. nerdy), we probably don't have to worry about crime, bankruptcy, or huge red marks on our history. BUT we are at risk for probably having too much "foreign influence." Most of us, being international affairs-minded likely have family ties to other countries which sparked our interest in transnational affairs. Also, while some of the laws regarding weed are being relaxed, the federal law is still strongly in place. We are young and if it is harder to prove that any drug use (or any other crime/mischief) is "mitigated by time" since we simply do not have as much time on this earth as most other applicants. A few times a few years ago might fly, but I know this stuff is decided on a case by case basis. Having tried it once while younger though should not disqualify people like it might have several years ago.
If you think you have too much drug use or foreign influence, I just feel like it's fair that you are aware of these risks. Do not plan on lying during the security clearance, as the State Dept will most likely find out and the consequences will be far worse. NO LYING. NONE.
After you become a finalist you do the interview and writing exam process. Some days or weeks later you will be told if you are a winner. You have some time to respond to the offer. You are committed when you reply to accept the offer because you reply with your signed and notarized agreement. Once it is sent to them, I'm pretty sure that is when you can no longer back out (to answer a commenter's question).
All in all, the Pickering Fellowship is a fantastic opportunity and I truly feel gifted to be part of this community. There is a slight risk for financial hardship, though, if the security clearance is not obtained after the appeal. You will likely know, though, if your history might have too much foreign influence, drug use or other red flags. The contractual obligation is a large part of the fellowship, but is one we are happy and thankful we can fulfill.