Criteria for evaluating research on herbs and other natural products

[First published in Leung’s Chinese Herb News (LCHN-19), Mar/Apr, 1999; reprinted in My Life & Rollercoaster Career (MLRC), 2018, pp. 330-334]

Too many scientists and researchers investigating botanical medicines frequently treat herbal materials as if they were pure single-component drugs. This has resulted in countless numbers of publications that are meaningless (Issue 18), which in turn has wasted considerable amount of our precious resources and mental energy in disseminating and/or debunking.  To help scientists and writers/editors who are not familiar with the intricacies and complexities of natural product research, the following are some guidelines for evaluating and accepting natural products for study or manuscripts for publication. They also will serve as basic information for abstractors to include in their abstracts of published papers. I have divided them into 2 standard levels. The higher-level criteria should be ones whose attainment is our ultimate goal. With this higher standard, results of investigations in this field are more likely to be consistently duplicated. On the other hand, the minimal-level criteria are ones that should constitute the basic requirements for accepting a natural product for research or a manuscript for publication as well as minimal information to be included in abstracts. This lower standard is necessary for now because, at present, there are not too many publications that meet the ideal criteria.  However, as researchers not trained in the comprehensive aspects of natural products research get acclimated to this field, the ideal criteria naturally will then be adopted.

1. Commercial products without disclosure of formulas. Frequently, researchers publish reports based on a commercial or proprietary product, without revealing what the product is. In the Chinese herbal/medical literature, there are many publications of this type. The information in them is meaningless and useless, except to manufacturers and marketers of the investigated products.

Minimal (to allow traceability):

• Name and address of manufacturer

• Concentration(s) used in the study

• Method(s) of administration

• Source of financial support if other than manufacturer/marketer

Ideal:

• Reject the material or manuscript

2. Pure natural compounds. They should be treated as any pure natural

chemicals (e.g., caffeine, ephedrine, huperzine A, synephrine), with indication of whether they are isolated from plants or chemically synthesized.

Minimal:

• Chemical name

• Purity

• Concentration(s) used in study

• Method(s) of administration

Ideal (all above, plus):

• Plant source (Latin binomial), with authenticating authority

• Plant part (s), with authenticating authority

3. Purified extracts containing artificially high concentrations of specific chemical compounds or groups of chemicals. They include extracts of green tea with high amounts (e.g., 90%) of certain polyphenols (catechin, epigallocatechin, epigallocatechin gallate, etc.), of Asian ginseng with high total ginsenoside content (e.g., 80-90%), of grape seed or pine bark with high proanthocyanidin content (e.g., >80%), and of milk thistle with silymarin. Since the contained chemicals are present in such artificially high levels, they no longer bear resemblance to the botanicals from which they are extracted. These extracts are the ones that can cause the most problems. Unless the whole extraction process (including solvents) is revealed, there is no easy way to ascertain, besides the named chemicals (markers or actives), what else is present in the extract. For example, does the remaining part of the extract contain other even more active components from the botanical drug, or is it made up of only excipients? What is the chemical profile of the extract? Is this chemical profile consistent and how comparable is it to ones previously reported? Variations among these factors can greatly affect the biological activities of these extracts.  The more precisely we identify these parameters, the more likely can the results be duplicated by future studies. To perform scientific studies on these natural materials without addressing these issues would not yield consistent and meaningful results.

Minimal:

• Plant source(s) (Latin binomials), with authenticating authority

• Plant part(s), with authenticating authority

• Percent purity of marker(s)/active(s) in extract

• Chemical profile of marker(s)/active(s) (minimum 2 of: HPLC,

TLC, GC, etc.)

• Concentrations used in study

• Method(s) of administration

Ideal (all above, plus):

• Nature of extract (solvents used and ratios)

• Total chemical profile of extract (minimum 2 of: HPLC, TLC, GC, etc.)

• Excipients used in extract

4. Standardized extracts. These are extracts with a standardized amount of one or more marker or active compounds. There are 2 major types: total extracts containing specified amounts of markers or actives plus other compounds also naturally present; and partial extracts containing specified amounts of markers and actives, but lacking other components present in total extracts. As with purified extracts containing high concentrations of specific markers or active compounds, the same types of issues relating to solvents used and consistency of chemical profile apply.

Minimal:

• Plant source(s) (Latin binomials), with authenticating authority

• Plant part(s), with authenticating authority

• Percent purity of marker(s)/active(s) in extract

• Total chemical profile of extract (minimum 1 of: HPLC, TLC, GC, etc.)

• Concentrations used in study

• Method(s) of administration

Ideal (all above, plus):

• Nature of extract (solvents used and ratios)

• Chemical profile of marker(s)/active(s) (minimum 2 of: HPLC,

TLC, GC, etc.)

• Total chemical profile of extract (1 more of: HPLC, TLC, GC, etc.)

• Excipients used in extract

5. Regular extracts. These are extracts with no standardized amounts of marker or active compounds. Their strength may be expressed in ratios between raw herbs and extracts (e.g., 4:1, meaning 1 kg of extract is derived from 4 kg of raw herb) or as percent of herb material in a specific solvent (e.g., 20% extract in 70% ethyl alcohol, meaning 100 g or mL of the hydroalcoholic extract is derived from 20 g of crude herb). However, these strengths are meaningless unless solvents used in their extraction are given. For example, a strength of 10:1 to describe extracts of astragalus root or Asian ginseng root is meaningless, unless the solvent(s) are clearly stated, because a normal exhaustive extraction of either herb with water will result in extracts of no more than a 3.5:1 strength. On the other hand, an extraction with 1-butanol would yield very little extractives and thus would result in extracts of high strength (e.g., 10:1). However, these extracts do not represent these botanicals in traditional properties or in chemical profiles.

Minimal:

• Plant source(s) (Latin binomials), with authenticating authority

• Plant part(s), with authenticating authority

• Type of extract (tincture, fluid extract, solid extract, powdered

extract, etc.)

• Solvent(s) used and ratios

• Strength (ratio of crude herb to extract)

• Concentration(s) used in study

• Method(s) of administration

Ideal (all above, plus):

• Total chemical profile of extract (minimum 2 of: HPLC, TLC, GC, etc.)

• Dosage form used (tablets, capsule, syrup, drink, etc.)

6. Crude botanicals. Sometimes powdered herbs and fresh herbs or juices are used in studies. It is important to be sure the following minimum information is provided.

• Plant name(s) (Latin binomials), with authenticating authority

• Plant part(s), with authenticating authority

• Form used (fresh, juice, dried, dried after processing, etc.)

• Dosage form used (capsule, tablet, drink, etc.)

• Method of administration or application (oral, topical, etc.)

• Amount(s) used in study

The above guidelines I have provided are by no means complete. But at least they can serve as a start. I am sure some of my esteemed colleagues who are well versed in this field will provide further suggestions and comments. However, there are several caveats. Thus, despite all these criteria, an uninformed investigator could always provide a plant name (Latin binomial) even though he/she may have no idea of its authenticity. Consequently, it is imperative that the authority who authenticated the plant material be identified in the publication. Also, fundamental problems relating to the influences of growing location, time of harvest, and age of plant at harvest, as well as other geographical and climatic factors, need to be addressed on an ongoing basis until resolution is achieved.  I am not the only scientist who sees as a major threat to natural product research, the use of dubious plant materials, which leads to the proliferation of published information that is biased, dubious, and often plain wrong. As the few examples described in the last issue of this newsletter [Issue 18] demonstrate, we, as responsible scientists, must take the challenge and responsibility to stop this “cancer” that is growing out of control. We need to have relevant organizations such as the American Society of Pharmacognosy (ASP) take the lead in refining these guidelines and promoting their adoption by fellow scientists. ASP should itself encourage its own members to follow them as well as enforcing them in its own publication and publications of its sister organizations. If we, as a small group of scientists who understand the complexities of natural product research, do not take the lead, the scientific and medical fields would be drowned in quasi-scientific herbal gibberish in 10 years. Just look at the sudden proliferation of books, journals, magazines, and newsletters on this subject over the past 5 years! Too much damage has already been done!

From my Newsletter #37, Mar/Apr 2002, Reprinted in My Life & Rollercoaster Career (2018, pp. 466-472)

From my Leung’s (Chinese) Herb News (LCHN-37, Mar/Apr 2002), reprinted in My Life & Rollercoaster Career  (MLRC 2018, pp. 466-472); the following is just one of 3 topics and on pp. 470-472:

HOW TO IMPROVE THE QUALITY OF SCIENTIFIC DATA ON HERBS

In order to do this, we have to address the problem at 3 levels:

(1) research; (2) publication; and (3) abstracting, indexing, and data input into databases.

I am not too concerned about basic scientific research technologies. We all learned those in college and then graduate school, and further honed our skill in our ‘real’ research jobs. What I am concerned about is that most researchers who have not been trained in natural medicines don’t seem to have a feel for the importance of the clear definition of these materials before using them as objects of research. Although we have all been trained in the use of good science in our research, we often ignore it when it comes to herb research. We all know the importance of applying the right quality control to the research material (e.g., a chemical or drug substance) and would never accept one that is not well defined. For instance, we would never think of investigating the effects of alcohol on, say, human cognitive function, by using a liquid called ‘alcohol.’ Based on our training, it is almost second nature for us to clearly define the alcohol (in this case, obviously ethanol, because it’s not ethical or permissible to use other alcohols on human experimentation), its concentration, purity (does it contain toxic adulterants?), etc. We would never accept just any ‘alcohol’ because it is an ‘alcohol’ or close enough, such as methyl alcohol or isopropyl alcohol. Yet many research studies on herbal medicines/supplements use test materials that are vaguely defined, if at all. Terms such as “ginseng,” “echinacea,” “kava kava,” and “St. John’s wort” have been routinely used alone as sole description of the test materials used in the studies, in at least 1 of the 3 levels mentioned above. In some of the better-defined (but still grossly inadequate) materials used, researchers may use the Latin binomials of the plant species (from which the material is derived), believing that that would be the definitive proof of identity. However, without providing which part of the plant supplies the test material, the scientific name of the plant is meaningless, and the study using this still-basically-undefined material will produce dubious and meaningless results. For example, again using hawthorn, how meaningful would the study of “Crataegus monogyna Jacq.” be? Here we have ‘scientifically identified’ English or one-seed hawthorn. But it has no relevance to the material under study. Is it the leaf, the seed, the fruit, the flowers, the root, or a combination of these parts? Also, is it the powdered crude material (whatever plant part) or is it an extract in water, ethanol (%?), or other organic solvent? And if so, how concentrated is it? Also, is it standardized to chemical markers, and which ones? These are just a few variables that must be clearly defined for resulting data to be reproducible and usable. Amazingly, many reports still simply use the name of the species (e.g., ‘hawthorn’ or ‘Crataegus monogyna’) to describe the materials used in the studies, at least in level 3 and sometimes also in level 2. No wonder we are suddenly being inundated with so much herb research data that are dubious or ambiguous and controversial! Which reminds me of how the use of Latin binomials to define a plant species is only as valid as the competence of the person who uses it. I used to know a flavor chemist who was in charge of the laboratories of an extraction company. He had the Latin binomials of most plant species associated with the materials being extracted at his plant memorized, though he had no clue as to how to identify the plant materials other than a few very common ones used in flavors, such as fenugreek seed, carob pod, chili pepper, coffee bean, etc. He was quick to put in the Latin binomials on his certificates of analysis, even though he might have no idea whether the materials extracted were actually from the plant species he put down on his certificates of analysis. Yet he was very proud of the fact that he knew the Latin binomials of plants and followed ‘standard scientific protocol’ when making his reports (certificates of analysis). To me, this is a typical case of a theoretical intention that does not always lead to a proper successful execution. Furthermore, in this case, it gives one a false sense of security by believing a correctly identified test material has been used. I suspect this situation is not unique. It probably applies to many companies involved in herb research and/or manufacture as well as to academia and other research institutions. The key to improving the quality of scientific herb research data lies in the clear definition of the test materials (crude botanicals and different forms of extracts) used in any research.

Currently, there are no official or universally recognized guidelines (or criteria) for defining such materials. The late Dr. Varro (Tip) Tyler and I have independently addressed this issue (Issues 19 & 35 160). In my article, I have actually provided guidelines for defining these test materials, which can and should be used at all 3 levels. Those criteria were first published in the March/April 1999 issue of this Newsletter and later reprinted in the November/December 2001 issue. Yet to date, none of the journals in our fields (pharmacognosy, natural products, phytomedicine, herbal medicine, Chinese medicines, ethnomedicine, etc.) which should be the leaders in herb research, has set minimal criteria for defining test materials as conditions in accepting manuscripts for publication. I just don’t get it. How can my colleagues, who are publishers, editors, or reviewers of these journals and who are supposed to be experts in this field, continue to allow the publication of herb research data that are often not worth the paper on which they are printed?

Furthermore, the longer these publications are allowed to continue to inundate us with ambiguous and dubious (some egregious) information, the more expensive and difficult it will be for us to rectify the problem. Without reliable scientific research data on which to base further research on botanical medicines or dietary supplements, we will continue to generate irreproducible, ambiguous, dubious, and, yes, controversial data. And we will be wasting our taxpayers’ money by continuing to support such research. The end result would be a declaration by the pharmaceutical and medical professions telling the world ‘I told you so,’ supported by scientific ‘evidence’ from research sponsored by our government. The only way to rectify this whole mess and stop wasting any more money and energy (first to produce dubious/ambiguous data and then try to deal with the controversy and more misinformation generated by these data) is to immediately institute criteria or guidelines for researchers, journal editors/reviewers, and data entry professionals (abstractors, indexers, database managers) to clearly define test materials before being accepted for research, publication, and/or incorporation into databases. These criteria should be instituted at all 3 levels – research, publication, and database. To start, they don’t need to be all-at-once comprehensive. The most urgent need is for these criteria to be there so that professionals at all 3 levels of the information generation and dissemination chain will be aware of the futility and wastefulness of dealing with undefined research materials.

The key is to get them to stop equating a natural test material to a pure chemical drug or a plant species (common or scientific name), and to start thinking about what that test material actually is or should be. This will be the only way to ensure that the information generated from botanical medicine/supplement research has a universally acceptable level of quality, which scientific researchers from healthcare and related fields may use with confidence to develop new, credible, and useful data.161

160 Tyler, V.E., Scientific Review of Alt. Med. 4(2): 17-21(2000).

161 C.A. Swanson, “Suggested guidelines for articles about botanical dietary supplements,” Am J Clin Nutr 2002; 75: 8-10.;