This is an open access article published under an ACS AuthorChoice License, which permits
copying and redistribution of the article or any adaptations for non-commercial purposes.
Cite This: ACS Nano 2018, 12, 7434−7444 www.acsnano.org
How To Optimize Materials and Devices via
Design of Experiments and Machine Learning:
Demonstration Using Organic Photovoltaics
Bing Cao,†,‡ Lawrence A. Adutwum,*,†,§,‡ Anton O. Oliynyk,†,‡ Erik J. Luber,†,‡
Brian C. Olsen,*,†,‡ Arthur Mar,*,†,‡ and Jillian M. Buriak*,†,‡
†Department of Chemistry, University of Alberta, 11227 Saskatchewan Drive, Edmonton, AB T6G 2G2, Canada
‡National Institute for Nanotechnology, National Research Council Canada, 11421 Saskatchewan Drive, Edmonton, AB T6G 2M9,
Canada
§Department of Pharmaceutical Chemistry, College of Health Sciences, University of Ghana School of Pharmacy, P.O. Box LG 43,
Legon, Ghana
*S Supporting Information
ABSTRACT: Most discoveries in materials science have been made empirically, typically through one-variable-at-a-time
(Edisonian) experimentation. The characteristics of materials-based systems are, however, neither simple nor
uncorrelated. In a device such as an organic photovoltaic, for example, the level of complexity is high due to the
sheer number of components and processing conditions, and thus, changing one variable can have multiple unforeseen
effects due to their interconnectivity. Design of Experiments (DoE) is ideally suited for such multivariable analyses: by
planning one’s experiments as per the principles of DoE, one can test and optimize several variables simultaneously, thus
accelerating the process of discovery and optimization while saving time and precious laboratory resources. When
combined with machine learning, the consideration of one’s data in this manner provides a different perspective for
optimization and discovery, akin to climbing out of a narrow valley of serial (one-variable-at-a-time) experimentation, to a
mountain ridge with a 360° view in all directions.
The recent rise of materials informatics and machine- informatics can provide scientific insights, thus enhancinglearning-based computational discovery of materials, chemical intuition. Machine-learning models are becomingsuch as the Materials Genome Initiative, provides new more available and user-friendly, whether or not researchers
sources of fuel (ideas, directions) for materials science.1−5 The have a background in informatics (e.g., Citrine’s materials
ability to screen millions or more possible candidates informatics, MI, platform6). The materials informatics
computationally for a given application or set of properties is community is working toward the principle of inclusivity,
resulting in new leads for experimentalists as vast parameter making tools available for the community, such as the
spaces are screened, opening up previously unconsidered Matminer MI library,
7 Magpie materials descriptors,8 scikit-
9
avenues of research. One of the most important features of learn machine-learning package, and COMBO Bayesian
10
machine-assisted methods is the ability to predict a wide range optimization library. These tools have been used extensively
of materials properties, even when fundamental understanding in materials science and engineering to target novel materials
of the chemistry or physics behind the property is lacking.
Even when fundamentals are well-understood, materials Published: July 20, 2018
© 2018 American Chemical Society 7434 DOI: 10.1021/acsnano.8b04726
ACS Nano 2018, 12, 7434−7444
Downloaded by 197.255.69.90 at 03:21:23:178 on July 01, 2019
from https://pubs.acs.org/doi/10.1021/acsnano.8b04726.
Perspective
ACS Nano Perspective
and optimization strategies and to tackle classification well as optimizing conditions during the development of
problems.11−22 established materials.
Lagging behind machine-learning-based discovery is the
experimental side of the equation, leading to the following In this Perspective, we describe how
question: Can experimental optimization keep pace with
machine learning? Edisonian or empirical screening of new Design of Experiments, combined with
materials or devices in an experimental laboratory takes machine-learning analysis, can dramat-
considerable time (time scales of months to years) and ically increase the rate of screening and
resources (many thousands of dollars for salaries, supplies, and optimization of materials properties
instrument time).23 Even if machine learning suggests a new
family of materials with given properties, synthesis and and devices.
optimization in the laboratory requires consideration of
experimental parameters not limited to precursor choice, The basic ideas behind DoE date back to the early 20th
synthetic method, temperature, atmosphere, molar ratios, century, with the work of British statistician and geneticist,
additives, and many others. Chemical intuition is valuable Ronald Fisher, who wrote The Design of Experiments, published
but is also convoluted with preconceived bias and, in a in 1935.26 The chemical statistician, George Box (who
multidimensional system with many variables, may be flawed. happened to be married to Fisher’s second daughter), was
greatly influential throughout the second half of the 20th
century with his work on statistical modeling and DoE, among
many other topics.27−32 DoE has since demonstrated its utility
in manufacturing, engineering, and other fields.33 Several
excellent tutorials outlining the use of DoE for analytical34,35
and pharmaceutical24 chemistry have been written. In this
Perspective, we describe the fundamentals of DoE for materials
optimization and provide an example of optimization of the
bulk heterojunction of organic photovoltaic (OPV) devices
through two rounds of parameter optimization. We will
demonstrate the enormous potential for experimentalists in
materials science to arrive at “real” optima more rapidly, be it
device performance or another characteristic. By applying
machine-learning methods to the results of DoE, multidimen-
sional maps can be rendered to enable the experimentalist to
Figure 1. So many optionshow do we choose? The authors see, without bias or preconceived ideas, not only the areas or
pondering a small subset of the myriad of published components domains of “best” performance but also new areas that have
that have been tested in organic photovoltaic (OPV) devices. not been previously considered. Design of Experiments,
Image credit: Kelli Luber. combined with machine-learning methods, is a powerful tool
not only to improve your system or materials in a much more
directed manner but also to elevate the experimentalist to be
In academic laboratories, we typically teach our students to able to view the landscape of possibilities clearly and to make
change one variable at a time in order not to confound the new discoveries that had previously been obscured.
roles of different factors that lead to the observed result. This Fundamentals of Design of Experiments (DoE):
approach to experimentation is termed one factor/one variable Optimization of Multiple (Dependent and Independ-
at a time (OFAT, or OVAT) and is limited not only by speed, ent) Variables. Take a simple material or a device to optimize
as it is slow, but it also rarely results in discovery of optima.24 that has two uncorrelated parameters, like the one seen in
The sheer number of variables requires the researcher to make Figure 2a,b. The blue cloud represents the region of best/
educated guesses throughout, and this process infrequently optimum performance that the experimentalist would like to
leads to truly optimized parameters.25 Put more succinctly, one find. Optimizing one variable at a time (OFAT or OVAT), one
cannot even know if one’s “best” results are indeed the overall of the two parameters would first be chosen to be screened
best that can be achieved with the material or device at hand, (green line), and then using that optimum on the green line as
as vast areas of parameter space would have been missed. In the starting point for orthogonal screening of the second
this Perspective, we describe how Design of Experiments, variable, the experimentalist would then optimize in an
termed DoE, combined with machine-learning analysis, can orthogonal direction (orange line). In this simple example,
dramatically increase the rate of screening and optimization of with a sequential experimental approach, the experimentalist
materials properties and devices. Several variables can be would find the optimum as this approach does locate the
changed and investigated simultaneously if carried out in an center of the blue region that represents best performance or
appropriate statistical framework that is determined before characteristics. For more complex systems that have a greater
starting experiments, which can lead to much faster screening number of parameters or correlations, this simple experimental
of experimental parameters and optimization. The DoE approach would require a larger number of experiments and
approach enables the experimentalist to sample a large, yet could miss the optimum, as demonstrated in Figure 2c,d. In
multidimensional parameter space in a rational manner, the first series of tests (shown in green), the experimentalist
which can then be coupled with machine learning to map observes a maximum, but unlike the previous uncorrelated
approximately the parameter space. This methodology can system shown in Figure 2a,b, building from this maximum for
enable faster and higher fidelity exploration of new systems, as the orthogonal second set of experiments (orange line) does
7435 DOI: 10.1021/acsnano.8b04726
ACS Nano 2018, 12, 7434−7444
ACS Nano Perspective
find the system optimum, with some degree of confidence that
they have done so.
When moving away from OFAT experimental designs, one
must think about how to sample the parameter spacing
effectively when using DoE. The most naive method would be
to implement a full factorial design (Figure 3a), where all
Figure 3. Examples of factorial sampling in 2-factor 4-value system
showing (a) full factorial, (b) fractional factorial, and (c) Latin
square.
possible combinations of parameters are tested. Although this
approach would yield the highest fidelity approximation of the
behavior of the desired output parameter, it would typically
require an unfeasibly large number of experiments, which
Figure 2. Optimization of a two-parameter system: Design of scales exponentially with the number of parameters [number of
Experiments (DoE) vs one-factor-at-a-time. (a,c,e) Map of the true experiments = levels
(number of parameters or factors)]. As such, an
values (blue gradient) of an output parameter of interest (e.g., alternative approach would be a fractional factorial design,
yield, power conversion efficiency, polydispersity, etc.) as a where the parameter space is sampled in a checkerboard-like
function of the two input parameters (horizontal and vertical manner (Figure 3b), reducing the number of experiments by a
axes). (a,b) One-factor-at-a-time sampling of an uncorrelated system factor of 2 (this concept can be extended to an arbitrary
will generally lead to finding the optimum value of the output number of dimensions). If experimental constraints require
parameter. (c,d) One-factor-at-a-time sampling of a correlated even sparser sampling, fractional factorial design can be used to
system will generally not result in finding the optimum value of the reduce the number of experiments by a factor of 3 or 4. Given
output parameter. (e) DoE approach with orthogonal sampling of
a correlated system. (f) Approximation of the true output the exponential scaling of factorial designs (full or partial),
parameter map produced by machine-learning tting methods these methods can be overly expensive during the first round offi
applied to the six data points acquired by DoE in (e). parameter testing but become useful implementations upon
secondary rounds of optimization when a smaller parameter
space is being sampled. In the first round of optimization,
which often consists of a large number of parameters and levels
not yield the optimum value for the system. The experimen- being tested, an efficient sampling method to use is the Latin
talist could be self-deceived into thinking that they had found square (Figure 3c). This sampling technique tests every value
the best performance, when in fact, they did not. This method from each parameter only once, enabling one to see if there are
cannot reveal if the actual maximum has indeed been found particular correlations between the variables and the individual
because a simple OFAT optimization approach reveals little effects on the experimental results.
about correlations between the system parameters. Consider Example of Design of Experiments Applied to
the case where the experimentalist tackles the same correlated Organic Photovoltaics. Organic photovoltaic (OPV)
system, but instead of choosing points on lines via serial technologies are of great interest because of the potential to
optimization, they choose specific points in parameter space mass manufacture these “plastic” solar cells through ambient
using DoE principles, where the points are distributed processing, such as roll-to-roll printing, inkjet printing, and
orthogonally and evenly like those found in Figure 2e. With spray coating.36,37 Unlike silicon-based solar cells that have an
the data from these points, data analysis techniques from energy payback period of several years, organic photovoltaics
machine learning can be used to build a map of the whole could theoretically have an energy payback as short as 24 h due
parameter space (Figure 2f), instead of only having linear to the low energy costs associated with their production.38
series of data along a small number of lines (directions). This Organic photovoltaics also have the advantages of being
map from the first six experiments also does not currently lightweight and flexible39 with tunable color and the potential
contain the maximum, but it shows an area of interest and for recycling. Power conversion efficiencies (PCEs) of OPV
reveals correlations between the parameters. At this point, the devices have now reached over 14%, but these solar cells are
experimentalist would then use this map to devise a second set not yet commercialized.40 Organic photovoltaic devices suffer
of experimental points focused on the area of interest; the from problems of instability (thermal, photochemical, and a
ensemble of experiments needed to reach this point is not combination thereof), batch-to-batch variability of the polymer
greater than the number of experiments carried out in Figure components, and subtle but important variations of solution-
2a−d. Continuing in this way, the experimentalist will likely processing parameters.41
7436 DOI: 10.1021/acsnano.8b04726
ACS Nano 2018, 12, 7434−7444
ACS Nano Perspective
Figure 4. (a) Schematic of single junction organic photovoltaic (OPV) devices, showing the bulk heterojunction (BHJ; in red), and the
multiple interfacial layers in the device. (b) Schematic of BHJ morphology: in this case, a low band gap polymer donor and a fullerene
acceptor undergoing nanoscale phase segregation into discrete nanoscale domains of donor and acceptor. The use of an additive is often
purported to assist in nanodomain formation, as shown here. Images reprinted from refs 42 and 43. Copyrights 2016 and 2008, respectively,
American Chemical Society.
Organic photovoltaic devices have a complex sandwich cannot know if an obtained optimum is indeed the best
architecture (Figure 4). The “meat” of the sandwich is called possible performance of the device. A many-dimensional
the bulk heterojunction (BHJ), a mixture of two or more analysis is needed as there are so many variables to consider.
phase-segregated materials that comprise polymers or small Thus, a DoE approach is ideal, if not essential, to determine
molecules.44 The BHJ absorbs light and results in an exciton which variables are important and, then, using those important
that separates into an electron−hole pair. Many hundreds, and variables, to find the optima.
almost certainly thousands, of BHJs have been tested and
published for OPV performance, and the efficiencies of these The use of Design of Experiments, with
devices depend precisely upon the morphology of the BHJ that its intentional exclusion of precon-
results from the processing parameters chosen.36,45−47 The two
charge carriers formed in the BHJ, the electron and the hole, ceived bias or notions, enables us to
must then travel to their respective electrodes, a process that is “see the big picture”before, the
assisted by interfacial layers on the electrodes that may experimentalist was walking through a
specifically transport (or block) one of the charge carriers or narrow valley of one-dimensional data,
produce beneficial compositional gradients in the BHJ, among
a host of other effects. Many interfacial layers have been tested but these maps are the equivalent of
in both forward and reverse configurations;48 the decision tree viewing the landscape from a moun-
to arrive at the/an ideal layer(s) and configurations is not tain ridgetop.
obvious,42,49 meaning empirical screening is typically the
primary route. The electrodes themselves have traditionally Here, we outline the process of applying a DoE approach
been a transparent conducting oxide such as indium tin oxide toward the optimization of a BHJ, the core of an OPV device,
and a thin reflective metal back contact, but now there is a which comprises a low band gap polymer, poly[N-9′-
large and growing number of electrodes based upon metal heptadecanyl-2,7-carbazole-alt-5,5-(4′,7′-di-2-thienyl-2′,1′,3′-
nanowires and thin metal films with plasmonic and light benzothiadiazole)] (PCDTBT). This polymer is a well-
trapping properties, and others.50 established low band gap polymer with a reported PCE
The array of choice with respect to potential components in range from 3.0−6.0% in the literature in the standard, forward
a particular OPV device is vast, and it is physically impossible solar-cell structure (ITO/PEDOT:PSS/BHJ/LiF/Al).58−60 In
to screen every combination, or even a small subset, for every this example, we started with the “standard” PEDOT:PSS/ITO
possible architecture due to a lack of time and resources. The electrodes and focused on applying a DoE approach to
number of combinations increases even further in the case of optimize the PCE of these BHJ devices. The characteristics of
tandem architectures.51−53 In addition, the role of the the BHJ examined included the ratio of donor to acceptor, the
morphology of the BHJ is now well-established to be critical thickness of the BHJ layer, and the concentration of a common
for device performance and is exquisitely dependent upon the additive, diiodooctane; these seemingly simple parameters play
processing parameters, which, if carried out from solution, critical roles in the nanoscale phase segregation of the layer
implies further decisions regarding concentration, ratio of and, thus, the resulting device efficiency. The thickness of the
donor to acceptor, choice and quantity of additives, annealing BHJ depends upon both the solution concentration and spin-
procedures (thermal, solvent, or both), and choice of casting speed, and thus, the four factors selected for the first
solvent(s).54−57 Empirical screening is the typical route to a round of optimization are summarized in Table 1. There are
determination of what the experimentalist establishes to be additional parameters that also could, and should, be
optimum conditions, but as shown visually in Figure 2 for a considered if time and resources permit expansion of the
simple, two-parameter optimization, missing the maximum is DoE parametrization to higher dimensions, such as post-spin-
possible, if not probable, due to the vast space of choice coating annealing parameters, choice of solvent for spin-
regarding materials and conditions. Worse, as mentioned coating, and temperature of the solution before spin-coating of
earlier, with an OFAT/OVAT approach to optimization, one the donor−acceptor solution, among many others.61−63 These
7437 DOI: 10.1021/acsnano.8b04726
ACS Nano 2018, 12, 7434−7444
ACS Nano Perspective
Table 1. Factor Selection for the First Round of Design of highly interdependent parameters render this system extremely
Experiments for the Optimization of PCDTBT:PCBM Solar difficult to optimize through the usual one-variable-at-a-time
Cells (OFAT/OVAT) approach.
For the four initial parameters chosen (Table 1) for DoE, it
parameters/factors parameter range levels is essential to select a wide range of values with as little
donor weight percentage (wt %) 10−55 4 prejudice as possible. The values should ideally result in
total solution concentration (mg/mL) 10−25 4 continuous BHJ films to enable testing of the resultant devices,
bulk heterojunction spin-cast speed (rpm) 600−3000 4 but one only anticipates good device performance from a small
processing additive (vol %) 0−12 4 subset of experiments; we knew from prior literature that some
of the combinations would almost certainly lead to low
parameters are almost certainly correlated to some degree and efficiencies, but inclusion of the fullest range of values leads to
thus make optimization of the BHJ system extremely difficult if more robust outcomes. This design, with four variables, is a
the experimentalist is changing only one parameter at a time. fraction factorial design representing 1/16 of the total
These principles would also, ideally, be applied to determine experiments for a full factorial designwith four factors (i.e.,
ideal interfacial layers and other aspects of the device. parameters) and four levels for each factor, a full factorial
First Round of Bulk Heterojunction Optimization by design would require 44 = 256 experiments (not including
Design of Experiments. The three main physical processing
parameters that can be adjusted during the BHJ processing repetition for statistical significance). The factor permutations
step of solar-cell production are the donor−acceptor ratio, the were based on a Latin square sampling technique, enabling us
solution concentration, and the spin speed. As will be shown to spread our 16 experiments judiciously over the chosen
here, the effects of these variables on device performance, as parameter space to approximate the functional dependence of
well as that of additives, are highly interdependent and the PCE on these input parameters.
complex. The thickness of a spun film is determined by the Table 2 shows the outcomes of the 16 experiments using our
spin speed, solvent vapor pressure and solution viscosity. As factorial design and the resulting PCEs. Only experiment 1-16
both the ratio of the donor−acceptor components and the failed due to the limited solubility of PCDTBT in
solution concentration can affect the solution viscosity, all d i ch l o robenz ene s p i n - c a s t i n g th e 25 mg/mL
three of these parameters affect the film thickness. The PCDTBT:PCBM solution with 55% PCDTBT was not
thickness of the BHJ is directly correlated with absorption, possible at room temperature. The analysis of the data from
which manifests itself on the short-circuit current (J ) of the this first round, summarized in Table 2, involved two steps.
 scdevice. The BHJ morphology which plays a critical role in Analysis of variance (ANOVA) was used to evaluate the
OPV performance, manifested through changes in the fill relative importance of each of the four parameters given by
factor (FF) and J is directly influenced by spin speed and their percent contribution to the PCE of the OPV devices. Ansc
film thickness, both of which affect drying times.64 Drying time ANOVA compares the variance of the output, in this case
is important because nanoscale phase segregation occurs when PCE, for each of the input parameters. If the variance in the
the film is wet by solvent or additives. Total concentration and output is high for a particular parameter, then that parameter
the donor−acceptor ratio are also involved in the resulting has a high contribution. These contributions can be seen in
morphology of the BHJ film. Another popular approach to Figure 5. For more details on how to perform an ANOVA,
affect the BHJ morphology is the use of a low-vapor-pressure please see the Supporting Information. The total solution
additive such as diiodooctane in the BHJ solution to increase concentration and donor weight percentage contribute about
drying times. This additive, however, also affects solution 45 and 28%, respectively, to the resulting PCE, whereas
viscosity, meaning that its actual role is multifaceted.54,56 These additives only account for a contribution of less than 5%. The
Table 2. Summary of Parameters Used for PDCTBT:PC71BM Solar Cells in the First Round of Design of Experiments
Optimization, Resulting Power Conversion Efficiency (Uncertainty Expressed as the Standard Deviation), and Number of
Devices Prepared
experiment # donor % (wt %) total concentration (mg/mL) spin speed (rpm) additive (vol %) PCE (%) number of devices
1-1 10 20 3000 2 0.05(5) 14
1-2 10 25 1000 8 3.24(11) 10
1-3 10 10 600 0 0.016(16) 14
1-4 10 15 2000 12 0.0004(4) 10
1-5 25 20 600 12 7.14(13) 8
1-6 25 15 1000 2 3.22(32) 8
1-7 25 10 3000 8 0.00033(7) 14
1-8 25 25 2000 0 7.21(17) 11
1-9 40 10 1000 12 1.85(5) 3
1-10 40 20 2000 8 6.16(28) 12
1-11 40 25 600 2 3.90(8) 11
1-12 40 15 3000 0 2.27(35) 9
1-13 55 10 2000 2 1.16(4) 3
1-14 55 15 600 8 3.18(12) 10
1-15 55 20 1000 0 3.89(10) 13
1-16 55 25 3000 12 n/a n/a
7438 DOI: 10.1021/acsnano.8b04726
ACS Nano 2018, 12, 7434−7444
ACS Nano Perspective
lower spin speeds (lower values on x-axis). This area then
served as the basis for planning the next range of parameters to
be tested in the second round of optimization.
Second (Subsequent) Rounds of Parameter Evalua-
tion. Based on the results from the first round of optimization,
the parameters were further refined before a second round of
experiments was carried out. First, the additive parameter was
dropped as it contributed little to the PCE, as per the ANOVA
analysis (Figure 5 and Supporting Information). Second, the
ranges of each of the factors were narrowed. For instance, the
range of the donor−acceptor ratio was examined in the range
of 20−27%, the total solution concentration to 20−25 mg/mL,
Figure 5. Analysis of variance and factor evaluation of the first and spin speed to 1000−2000 rpm from the wider ranges
round of optimization for PCDTBT:PC71BM bulk heterojunction
solar cells. shown in Table 1. As summarized in Tables 3 and 4, a partial
Table 3. Three Factors Considered in the Second Round of
low contribution of additives enabled us to drop this parameter Design of Experiments Parameter Selection for the
from the next round of testing and, hence, fit the current data Optimization of PCDTBT:PC71BM Solar Cells
with one less dimension (three dimensions instead of four) to
parameters level 1 level 2 level 3
find areas of interest for further optimization. The data were fit
using Scikit-learn,9 with a support vector machine (SVM) donor % (wt %) 20 25 27
using a radial basis function (RBF) kernel, a popular machine- total concentration (mg/mL) 20 23 25
learning algorithm.65,66 With an RBF kernel, the algorithm will spin speed (rpm) 1000 1500 2000
fit best to Gaussian-shaped features that would normally be
found in cases of optimization. For more information on this factorial design, this time with only three factors and a smaller
machine analysis, please see the Supporting Information to find range, provided the bounds of our area as well as an even
the code to reproduce these fits. distribution filling the space. Table 4 and Figure 7b reveal that
Now that the data have been fit using three parameters, we all experiments resulted in PCEs ranging from 6.3 to 7.8%,
can generate a three-dimensional map that represents an with the highest PCE of 7.8% obtained from experiment 2-5.
approximation of the PCE at any point in this space. In order Data Analysis and Visualization. As before, in order to
to visualize this space, we generate two-dimensional value visualize the results of the experimental data from Table 4,
maps from the three-dimensional space in the following SVM with RBF fitting was applied to several of the measured
manner. Slices are taken through the three-dimensional space device parameters shown in Figure 8. The rows of Figure 8
at certain intervals along one dimension, making a series of show the fitting for three measured parameterspower
two-dimensional maps of the PCE. Figure 6 shows these slices conversion efficiency, short-circuit current, and open-circuit
taken at the four donor concentrations (10, 25, 40, and 55 wt voltageas color maps where values of these parameters are
% of donor), with x- and y-axes showing spin speed and total indicated by the corresponding vertical color bars. Each row
concentration, respectively. The color gradient, scaled as contains the same 13 points (the actual results) plotted on the
indicated by the color bar on the right, and the contour lines three axes. The x- and y-axes correspond to spin speed and
map out the PCE fit from the data in the first round of total concentration, respectively. The donor concentration is
experiments. The points plotted on the map correspond to the shown in the three plots in each row as a slice of the RBF at 20,
experimental results. Even with the sparse number of data 25, and 27 wt %. It can be seen that the 20 and 27 wt % donor
points in this space, an area of interest (higher PCEs) around concentrations have more variability within the test range for
the 25% donor concentration (Figure 6b) can be seen in the all the measured parameters than those with 25 wt %. The 20
higher total concentration range (higher values on y-axis) and and 27 wt % donor concentrations also have their maxima on
Figure 6. Support vector machine/radial basis function fits of the power conversion efficiency measured from solar cells produced using the
15 different parameter combinations (seen here as the dots) in the first round of optimization. They are plotted as value maps of slices
through the three-dimensional fit, with spin speed on the x-axis, total concentration on the y-axis, and slices through the donor
concentration axis plotted in (a) 10 wt % donor, (b) 25 wt % donor, (c) 40 wt % donor, and (d) 55 wt % donor.
7439 DOI: 10.1021/acsnano.8b04726
ACS Nano 2018, 12, 7434−7444
ACS Nano Perspective
Table 4. Summary of Parameters Used for PDCTBT:PC71BM Solar Cells in the Second Round of Design of Experiments
Optimization, Resulting Power Conversion Efficiency (Uncertainty Expressed as the Standard Deviation), and Number of
Devices Prepared
experiment # donor % (wt %) total concentration (mg/mL) spin speed (rpm) PCE (%) thickness (nm) number of devices
2-1 20 20 1500 6.32(6) 73 5
2-2 27 20 1500 7.21(17) 77 11
2-3 20 25 1500 6.83(7) 126 6
2-4 27 25 1500 6.96(6) 131 6
2-5 20 23 1000 7.77(29) 109 4
2-6 27 23 1000 6.87(14) 136 4
2-7 20 23 2000 6.43(19) 76 8
2-8 27 23 2000 7.65(24) 88 7
2-9 25 20 1000 7.43(11) 115 4
2-10 25 25 1000 6.88(18) 135 8
2-11 25 20 2000 7.32(30) 104 7
2-12 25 25 2000 7.21(31) 126 8
2-13 25 23 1500 7.4(5) 129 7
sample thickness, but the thickness and the donor concen-
tration have the greatest effects on visible light absorption.
Figure 9 shows the RBF plot of thickness and donor
concentration and the relationship to PCE. The 13 points
plotted on this map are the same experimental points shown in
Figure 8 and can be matched by their colors. What the
experimentalist can learn from this figure is that there are two
promising areas that are both surprising and worthy of further
exploration. This map suggests that a wider range of donor
concentrations can lead to high PCEs than had previously been
“believed” or assumed.59,67 The use of DoE, with its intentional
exclusion of preconceived bias or notions, enables us to “see
the big picture”before, the experimentalist was walking
through a narrow valley of one-dimensional data, but these
maps are the equivalent of viewing the landscape from a
mountain ridgetop. New routes and features that were
previously unimagined become visible, which could open up
new possibilities for research.
Word of Caution: Here Be Dragons. Caution should be
taken when applying DoE and machine-learning statistics to
any set of parameters (Figure 10). When designing partial
factorial experiments, one must consider the correlation
between variables, as well as the variance in the range tested.
Organic photovoltaics tend to be good examples of devices
with parameter sets comprising smooth transitions with respect
to PCE and related parameters (Jsc, FF, Voc). Many systems in
chemistry would be expected to be similar, but if the area of
interest is very small within the range being tested, or the onset
quite sharp, then the risk is that the area of interest will be
missed by a partial factorial design. Even in Figure 9, we
Figure 7. Mean power conversion efficiency (PCE) of the cells observed that the area of high PCE in the 20 wt % donor area
from the first (a) and second (b) rounds of optimization. Error in the lower part of the figure is quite small, with the sampling
bars represent the standard deviation of the cells’ PCEs. The set almost missing this area entirely. Another note of caution is
dashed line at 6.2% represents the published values for the PCE of that many of the machine-learning fitting methods, such as
PCDTBT:PC71BM after traditional optimization in a standard
forward structure, without addition of a third component, special RBF, were designed for interpolation only and not for
electrode, or optical spacer.58−60 extrapolation, as these data fits would have little meaning
outside of the family of experimental data points. In our second
set of OPV solar optimization experiments, the parameter
the outer edge of the test range. These results indicate that space was not sufficiently large to encompass the entire area of
further testing could reveal a larger area of even higher interest, thus warranting further experiments to expand the
performing OPV devices. boundaries as mentioned previously. Given the current fit
To simplify these results further, the thicknesses of the (Figure 9), we have little reason to believe that the observed
samples were measured using contact profilometry. As PCE will drop as the devices with 25 wt % BHJ layer decrease
discussed earlier, all three of the test parameters affect the in thickness (the middle left area) as the fit shows because this
7440 DOI: 10.1021/acsnano.8b04726
ACS Nano 2018, 12, 7434−7444
ACS Nano Perspective
Figure 8. Support vector machine/radial basis function fits of the three experimental parameters of the second round of optimization: spin
speed on the x-axis, total concentration on the y-axis, and donor concentration plotted in the columns labeled as (a,d,g) 20 wt % donor,
(b,e,h) 25 wt % donor, and (c,f,i) 27 wt % donor. The rows represent three different parameters, (a−c) power conversion efficiency, (d−f)
short-circuit current, and (g−i) open-circuit voltage, measured from solar cells produced using the 13 different parameter combinations
(seen here as the dots).
Figure 9. Radial basis function visualization of measured cell
power conversion efficiency versus thickness and donor concen-
tration of the solar devices from Figure 8. Figure 10. Carta marina 1st ed. by Olaus Magnus, 1572. A good
map will show new areas to discover (high performing devices/
materials), but there are pitfalls and shortcomings to avoid
region is outside the experimental parameters. With a small (dragons). Source: National Library of Sweden (https://www.kb.se/hitta-och-bestall/om-samlingar-och-material/kartor.html).
number of experiments, we have explored a wide range of
space as well as mapped areas of interest, but there is still
further exploration that could be done both inside and outside scarce resources more effectively, time being one of the
the bounds of the current parameters. scarcest, and to have a higher probability of arriving at a true
optimum. With the traditional one-factor-at-a-time (OFAT or
CONCLUSIONS: LESS HOPING AND MORE KNOWING OVAT) linear approach to optimization, one cannot know if a
The DoE approach to materials optimization combined with true optimum was reached. Imagine the situation of a graduate
machine-learning analysis enables the experimentalist to use student spending several years in the laboratory designing,
7441 DOI: 10.1021/acsnano.8b04726
ACS Nano 2018, 12, 7434−7444
ACS Nano Perspective
synthesizing, and characterizing a new family of low band gap Erik J. Luber: 0000-0003-1623-0102
polymers or small molecules. With what little time is left of Brian C. Olsen: 0000-0001-9758-3641
their tenure in the laboratory, devices are prepared, typically Arthur Mar: 0000-0003-0474-5918
following published protocols that were optimized for different Jillian M. Buriak: 0000-0002-9567-4328
compositions and materials. Application of a DoE approach
could enable that student to optimize at least one or two Notes
rounds of OPV devices efficiently, or any other type of device, The authors declare no competing financial interest.
with less guesswork. The DoE approach enables less hoping
and more knowing in terms of finding a productive and ACKNOWLEDGMENTS
constructive direction forward. The subsequent application of
machine-learning methods enables visualization of the data in a This work was supported by Future Energy Systems of the
profoundly meaningful way, taking the experimentalist to a University of Alberta (https://futureenergysystems.ca; Grant
mountain top with a 360° view of the landscape, as opposed to Nos. T12-P04 and T12-P01), the Natural Sciences and
seeing it as a one-dimensional string of individual experiments. Engineering Research Council (NSERC, Grant Nos. RGPIN-
The ultimate key to progressing from the DoE approach of 2014-05195 and RGPIN-2018-04294), Alberta Innovates
laboratory experimentation to machine learning is the Technology Futures (Grant No. AITF iCORE IC50-T1
generation of large data sets, which is particularly challenging G2013000198), and the Canada Research Chairs program
in the case of devices (of all kinds, and not just OPV). (CRC 207142). We thank Dr. Mario Leclerc for synthesizing
Continuing with the example of OPV, many tens of thousands the PCDTBT polymer used in this study.
of OPV devices have been reported in the literature but
scraping these data with relevant experimental parameters is a REFERENCES
daunting data-mining problem. Each paper is written in a (1) Le, T. C.; Winkler, D. A. Discovery and Optimization of
different format by experimentalists in different laboratories Materials Using Evolutionary Approaches. Chem. Rev. 2016, 116,
using different parameters, and the experimental sections 6107−6132.
themselves are written using different terms and with different (2) Tabor, D. P.; Roch, L. M.; Saikin, S. K.; Kreisbeck, C.; Sheberla,
levels of completeness; therefore, it is unclear that meaningful D.; Montoya, J. H.; Dwaraknath, S.; Aykol, M.; Ortiz, C.; Tribukait,
data can be easily extracted. In addition, if thousands of OPV H.; Amador-Bedolla, C.; Brabec, C. J.; Maruyama, B.; Persson, K. A.;
devices have been reported, we can assume that the results of Aspuru-Guzik, A. Accelerating the Discovery of Materials for Clean
many low-performing devices were discarded and not Energy in the Era of Smart Automation. Nat. Rev. Mater. 2018, 3, 5−
publishedthese negative results would be extremely useful 20.
in the context of DoE. The DoE approach could therefore be a (3) De Luna, P.; Wei, J.; Bengio, Y.; Aspuru-Guzik, A.; Sargent, E.
step toward formalizing data collection in a rational and Use Machine Learning To Find Energy Materials. Nature 2017, 552,
standardized manner, if only starting small, within a single 23−27.
group or institution or through a collaboration. An organized (4) Kim, E.; Huang, K.; Saunders, A.; McCallum, A.; Ceder, G.;Olivetti, E. Materials Synthesis Insights from Scientific Literature via
approach to experimental design and data collection and Text Extraction and Machine Learning. Chem. Mater. 2017, 29,
reporting could serve as a starting point for real machine 9436−9444.
learning applied to devices such as OPVs. Such an approach to (5) Raccuglia, P.; Elbert, K. C.; Adler, P. D. F.; Falk, C.; Wenny, M.
making academic materials science more efficient could have B.; Mollo, A.; Zeller, M.; Friedler, S. A.; Schrier, J.; Norquist, A. J.
enormous implications, enabling us to solve problems that Machine-Learning-Assisted Materials Discovery Using Failed Experi-
threaten humanity’s very survival more rapidly, such as the ments. Nature 2016, 533, 73−76.
optimization of technologies that underpin the transition to a (6) O’Mara, J.; Meredig, B.; Michel, K. Materials Data Infra-
low-carbon world. structure: A Case Study of the Citrination Platform To Examine Data
Import, Storage, and Access. JOM 2016, 68, 2031−2034.
ASSOCIATED CONTENT (7) Ward, L.; Dunn, A.; Faghaninia, A.; Zimmermann, N. E. R.;
* Supporting Information Bajaj, S.; Wang, Q.; Montoya, J.; Chen, J.; Bystrom, K.; Dylla, M.;S
The Supporting Information is available free of charge on the Chard, K.; Asta, M.; Persson, K. A.; Snyder, G. J.; Foster, I.; Jain, A.
Matminer: An Open Source Toolkit for Materials Data Mining.
ACS Publications website at DOI: 10.1021/acsnano.8b04726. Comput. Mater. Sci. 2018, 152, 60−69.
Experimental details, materials, OPV device fabrication (8) Ward, L.; Agrawal, A.; Choudhary, A.; Wolverton, C. A General-
and characterization, details of Design of Experiments Purpose Machine Learning Framework for Predicting Properties of
(DoE), machine-learning algorithms and code, and Inorganic Materials. npj Comput. Mater. 2016, 2, 16028.
photovoltaic test results for all devices (PDF) (9) Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion,
B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V.;
AUTHOR INFORMATION Vanderplas, J.; Passos, A.; Cournapeau, D.; Brucher, M.; Perror, M.;Duchesnay, É. Scikit-Learn: Machine Learning in Python. J. Mach.
Corresponding Authors Learn. Res. 2011, 12, 2825−2830.
*E-mail: adutwum@ualberta.ca. (10) Ueno, T.; Rhone, T. D.; Hou, Z.; Mizoguchi, T.; Tsuda, K.
*E-mail: bcolsen@ualberta.ca. COMBO: An Efficient Bayesian Optimization Library for Materials
*E-mail: arthur.mar@ualberta.ca. Science. Mater. Discovery 2016, 4, 18−21.
*E-mail: jburiak@ualberta.ca. (11) O’Mara, J.; Meredig, B.; Michel, K. Materials Data Infra-
ORCID structure: A Case Study of the Citrination Platform to Examine Data
Import, Storage, and Access. JOM 2016, 68, 2031−2034.
Bing Cao: 0000-0002-3825-4669 (12) Ramprasad, R.; Batra, R.; Pilania, G.; Mannodi-Kanakkithodi,
Lawrence A. Adutwum: 0000-0001-6912-1001 A.; Kim, C. Machine Learning in Materials Informatics: Recent
Anton O. Oliynyk: 0000-0003-0732-7340 Applications and Prospects. npj Comput. Mater. 2017, 3, 54.
7442 DOI: 10.1021/acsnano.8b04726
ACS Nano 2018, 12, 7434−7444
ACS Nano Perspective
(13) Xue, D.; Balachandran, P. V.; Hogden, J.; Theiler, J.; Xue, D.; Using the Desirability Function in Analytical Methods Development.
Lookman, T. Accelerated Search for Materials with Targeted Talanta 2014, 124, 123−138.
Properties by Adaptive Design. Nat. Commun. 2016, 7, 11241. (36) Lu, L.; Zheng, T.; Wu, Q.; Schneider, A. M.; Zhao, D.; Yu, L.
(14) Balachandran, P. V.; Kowalski, B.; Sehirlioglu, A.; Lookman, T. Recent Advances in Bulk Heterojunction Polymer Solar Cells. Chem.
Experimental Search for High-Temperature Ferroelectric Perovskites Rev. 2015, 115, 12666−12731.
Guided by Two-Step Machine Learning. Nat. Commun. 2018, 9, 1668. (37) Zhang, G.; Zhao, J.; Chow, P. C. Y.; Jiang, K.; Zhang, J.; Zhu,
(15) Gaultois, M. W.; Oliynyk, A. O.; Mar, A.; Sparks, T. D.; Z.; Zhang, J.; Huang, F.; Yan, H. Nonfullerene Acceptor Molecules for
Mulholland, G. J.; Meredig, B. Perspective: Web-Based Machine Bulk Heterojunction Organic Solar Cells. Chem. Rev. 2018, 118,
Learning Models for Real-Time Screening of Thermoelectric 3447−3507.
Materials Properties. APL Mater. 2016, 4, 053213. (38) Espinosa, N.; Hösel, M.; Angmo, D.; Krebs, F. C. Solar Cells
(16) Oliynyk, A. O.; Sparks, T. D.; Gaultois, M. W.; Ghadbeigi, L.; with One-Day Energy Payback for the Factories of the Future. Energy
Mar, A. Gd12Co5.3Bi and Gd12Co5Bi, Crystalline Doppelgan̈ger Environ. Sci. 2012, 5, 5117−5132.
with Low Thermal Conductivities. Inorg. Chem. 2016, 55, 6625− (39) Li, Y.; Xu, G.; Cui, C.; Li, Y. Flexible and Semitransparent
6633. Organic Solar Cells. Adv. Energy Mater. 2018, 8, 1701791.
(17) Sparks, T. D.; Gaultois, M. W.; Oliynyk, A.; Brgoch, J.; (40) Li, S.; Ye, L.; Zhao, W.; Yan, H.; Yang, B.; Liu, D.; Li, W.; Ade,
Meredig, B. Data Mining Our Way to the Next Generation of H.; Hou, J. A Wide Band Gap Polymer with a Deep Highest
Thermoelectrics. Scr. Mater. 2016, 111, 10−15. Occupied Molecular Orbital Level Enables 14.2% Efficiency in
(18) Maier, W. F.; Stöwe, K.; Sieg, S. Combinatorial and High- Polymer Solar Cells. J. Am. Chem. Soc. 2018, 140, 7159−7167.
Throughput Materials Science. Angew. Chem., Int. Ed. 2007, 46, (41) Lee, E. K.; Lee, M. Y.; Park, C. H.; Lee, H. R.; Oh, J. H.
6016−6067. Toward Environmentally Robust Organic Electronics: Approaches
(19) Oliynyk, A. O.; Antono, E.; Sparks, T. D.; Ghadbeigi, L.; and Applications. Adv. Mater. 2017, 29, 1703638.
Gaultois, M. W.; Meredig, B.; Mar, A. High-Throughput Machine- (42) Cao, B.; He, X.; Fetterly, C. R.; Olsen, B. C.; Luber, E. J.;
Learning-Driven Synthesis of Full-Heusler Compounds. Chem. Mater. Buriak, J. M. Role of Interfacial Layers in Organic Solar Cells: Energy
2016, 28, 7324−7331. Level Pinning Versus Phase Segregation. ACS Appl. Mater. Interfaces
(20) Oliynyk, A. O.; Adutwum, L. A.; Harynuk, J. J.; Mar, A. 2016, 8, 18238.
Classifying Crystal Structures of Binary Compounds AB Through (43) Lee, J. K.; Ma, W. L.; Brabec, C. J.; Yuen, J.; Moon, J. S.; Kim, J.
Cluster Resolution Feature Selection and Support Vector Machine Y.; Lee, K.; Bazan, G. C.; Heeger, A. J. Processing Additives for
Analysis. Chem. Mater. 2016, 28, 6672−6681. Improved Efficiency from Bulk Heterojunction Solar Cells. J. Am.
(21) Oliynyk, A. O.; Adutwum, L. A.; Rudyk, B. W.; Pisavadia, H.; Chem. Soc. 2008, 130, 3619−3623.
Lotfi, S.; Hlukhyy, V.; Harynuk, J. J.; Mar, A.; Brgoch, J. Disentangling (44) Cheng, P.; Li, G.; Zhan, X.; Yang, Y. Next-Generation Organic
Structural Confusion Through Machine Learning: Structure Pre- Photovoltaics Based on Non-Fullerene Acceptors. Nat. Photonics
diction and Polymorphism of Equiatomic Ternary Phases ABC. J. Am. 2018, 12, 131−142.
Chem. Soc. 2017, 139, 17870−17881. (45) Song, J.; Zhang, M.; Yuan, M.; Qian, Y.; Sun, Y.; Liu, F.
(22) Oliynyk, A. O.; Mar, A. Discovery of Intermetallic Compounds Morphology Characterization of Bulk Heterojunction Solar Cells.
from Traditional to Machine-Learning Approaches. Acc. Chem. Res. Small Methods 2018, 2, 1700229.
2018, 51, 59−68. (46) Gedefaw, D.; Prosa, M.; Bolognesi, M.; Seri, M.; Andersson, M.
(23) Pels, K.; Dickson, P.; An, H.; Kodadek, T. DNA-Compatible R. Recent Development of Quinoxaline Based Polymers/Small
Solid-Phase Combinatorial Synthesis of β-Cyanoacrylamides and Molecules for Organic Photovoltaics. Adv. Energy Mater. 2017, 7,
Related Electrophiles. ACS Comb. Sci. 2018, 20, 61−69. 1700575.
(24) Trinh, T. B.; Upadhyaya, P.; Qian, Z.; Pei, D. Discovery of a (47) Yan, C.; Barlow, S.; Wang, Z.; Yan, H.; Jen, A. K.-Y.; Marder, S.
Direct Ras Inhibitor by Screening a Combinatorial Library of Cell- R.; Zhan, X. Non-Fullerene Acceptors for Organic Solar Cells. Nat.
Permeable Bicyclic Peptides. ACS Comb. Sci. 2016, 18, 75−85. Rev. Mater. 2018, 3, 18003.
(25) Weissman, S. A.; Anderson, N. G. Design of Experiments (48) Yin, Z.; Wei, J.; Zheng, Q. Interfacial Materials for Organic
(DoE) and Process Optimization. A Review of Recent Publications. Solar Cells: Recent Advances and Perspectives. Adv. Sci. 2016, 3,
Org. Process Res. Dev. 2015, 19, 1605−1633. 1500362.
(26) Fisher, R. A. The Design of Experiments, 9th ed.; Macmillan (49) Huang, L.; Wang, G.; Zhou, W.; Fu, B.; Cheng, X.; Zhang, L.;
Publishing Co., 1971. Yuan, Z.; Xiong, S.; Zhang, L.; Xie, Y.; Zhang, A.; Zhang, Y.; Ma, W.;
(27) Box, G. E. P.; Wilson, K. B. On the Experimental Attainment of Li, W.; Zhou, Y.; Reichmanis, E.; Chen, Y. Vertical Stratification
Optimum Conditions. In Breakthroughs in Statistics; Springer Series in Engineering for Organic Bulk-Heterojunction Devices. ACS Nano
Statistics; Springer: New York, 1992; pp 270−310. 2018, 12, 4440−4452.
(28) Janacek, G. Time Series Analysis Forecasting and Control. J. (50) Lu, H.; Ren, X.; Ouyang, D.; Choy, W. C. H. Emerging Novel
Time. Ser. Anal. 2009, 31, 303−303. Metal Electrodes for Photovoltaic Applications. Small 2018, 14,
(29) Box, G. E. P.; Hunter, S.; Hunter, W. G. Statistics for 1703140.
Experimenters: Design, Innovation, and Discovery, 2nd ed.; Wiley- (51) Andersen, T. R.; Dam, H. F.; Hösel, M.; Helgesen, M.; Carle,́ J.
Interscience, 2005. E.; Larsen-Olsen, T. T.; Gevorgyan, S. A.; Andreasen, J. W.; Adams, J.;
(30) Box, G. E. P. Science and Statistics. J. Am. Stat. Assoc. 1976, 71, Li, N.; Machui, F.; Spyropoulos, G. D.; Ameri, T.; Lemaître, N.;
791−799. Legros, M.; Scheel, A.; Gaiser, D.; Kreul, K.; Berny, S.; Lozman, O. R.;
(31) DeGroot, M. H. A Conversation with George Box. Statist. Sci. et al. Scalable, Ambient Atmosphere Roll-to-Roll Manufacture of
1987, 2, 239−258. Encapsulated Large Area, Flexible Organic Tandem Solar Cell
(32) Smith, A. F. M. George Edward Pelham Box. 10 October Modules. Energy Environ. Sci. 2014, 7, 2925−2933.
1919−28 March 2013. Biogr. Mem. Fellows R. Soc. 2015, 61, 23−37. (52) Lu, S.; Ouyang, D.; Choy, W. C. H. Recent Progress of
(33) Ilzarbe, L.; Álvarez, M. J.; Viles, E.; Tanco, M. Practical Interconnecting Layer for Tandem Organic Solar Cells. Sci. China:
Applications of Design of Experiments in the Field of Engineering: A Chem. 2017, 60, 460−471.
Bibliographical Review. Qual. Reliab. Eng. Int. 2008, 24, 417−428. (53) Li, M.; Gao, K.; Wan, X.; Zhang, Q.; Kan, B.; Xia, R.; Liu, F.;
(34) Leardi, R. Experimental Design in Chemistry: A Tutorial. Anal. Yang, X.; Feng, H.; Ni, W.; Wang, Y.; Peng, J.; Zhang, H.; Liang, Z.;
Chim. Acta 2009, 652, 161−172. Yip, H.-L.; Peng, X.; Cao, Y.; Chen, Y. Solution-Processed Organic
(35) Vera Candioti, L.; De Zan, M. M.; Caḿara, M. S.; Goicoechea, Tandem Solar Cells with Power Conversion Efficiencies > 12%. Nat.
H. C. Experimental Design and Multiple Response Optimization. Photonics 2017, 11, 85−90.
7443 DOI: 10.1021/acsnano.8b04726
ACS Nano 2018, 12, 7434−7444
ACS Nano Perspective
(54) Pivrikas, A.; Neugebauer, H.; Sariciftci, N. S. Influence of
Processing Additives to Nano-Morphology and Efficiency of Bulk-
Heterojunction Solar Cells: A Comparative Review. Sol. Energy 2011,
85, 1226−1237.
(55) Ma, W.; Yang, G.; Jiang, K.; Carpenter, J. H.; Wu, Y.; Meng, X.;
McAfee, T.; Zhao, J.; Zhu, C.; Wang, C.; Ade, H.; Yan, H. Influence
of Processing Parameters and Molecular Weight on the Morphology
and Properties of High-Performance PffBT4T-2OD:PC71BM Organ-
ic Solar Cells. Adv. Energy Mater. 2015, 5, 1501400.
(56) Zhao, J.; Zhao, S.; Xu, Z.; Qiao, B.; Huang, D.; Zhao, L.; Li, Y.;
Zhu, Y.; Wang, P. Revealing the Effect of Additives with Different
Solubility on the Morphology and the Donor Crystalline Structures of
Organic Solar Cells. ACS Appl. Mater. Interfaces 2016, 8, 18231−
18237.
(57) Babics, M.; Liang, R.-Z.; Wang, K.; Cruciani, F.; Kan, Z.;
Wohlfahrt, M.; Tang, M.-C.; Laquai, F.; Beaujuge, P. M. Solvent
Vapor Annealing-Mediated Crystallization Directs Charge Gener-
ation, Recombination and Extraction in BHJ Solar Cells. Chem. Mater.
2018, 30, 789−798.
(58) Blouin, N.; Michaud, A.; Leclerc, M. A Low-Bandgap Poly(2,7-
Carbazole) Derivative for Use in High-Performance Solar Cells. Adv.
Mater. 2007, 19, 2295−2300.
(59) Park, S. H.; Roy, A.; Beaupre,́ S.; Cho, S.; Coates, N.; Moon, J.
S.; Moses, D.; Leclerc, M.; Lee, K.; Heeger, A. J. Bulk Heterojunction
Solar Cells with Internal Quantum Efficiency Approaching 100%. Nat.
Photonics 2009, 3, 297−302.
(60) Alem, S.; Chu, T.-Y.; Tse, S. C.; Wakim, S.; Lu, J.; Movileanu,
R.; Tao, Y.; Beĺanger, F.; Deśilets, D.; Beaupre,́ S.; Leclerc, M.;
Rodman, S.; Waller, D.; Gaudiana, R. Effect of Mixed Solvents on
PCDTBT:PC70BM Based Solar Cells. Org. Electron. 2011, 12, 1788−
1793.
(61) Yi, Z.; Ni, W.; Zhang, Q.; Li, M.; Kan, B.; Wan, X.; Chen, Y.
Effect of Thermal Annealing on Active Layer Morphology and
Performance for Small Molecule Bulk Heterojunction Organic Solar
Cells. J. Mater. Chem. C 2014, 2, 7247−7255.
(62) Wang, J.; Liang, Z. Synergetic Solvent Engineering of Film
Nanomorphology To Enhance Planar Perylene Diimide-Based
Organic Photovoltaics. ACS Appl. Mater. Interfaces 2016, 8, 22418−
22424.
(63) Engmann, S.; Ro, H. W.; Herzing, A.; Snyder, C. R.; Richter, L.
J.; Geraghty, P. B.; Jones, D. J. Film Morphology Evolution During
Solvent Vapor Annealing of Highly Efficient Small Molecule Donor/
Acceptor Blends. J. Mater. Chem. A 2016, 4, 15511−15521.
(64) Huang, Y.; Kramer, E. J.; Heeger, A. J.; Bazan, G. C. Bulk
Heterojunction Solar Cells: Morphology and Performance Relation-
ships. Chem. Rev. 2014, 114, 7006−7043.
(65) Smola, A. J.; Schölkopf, B. A Tutorial on Support Vector
Regression. Statistics and Computing 2004, 14, 199−222.
(66) Chang, Y.-W.; Hsieh, C.-J.; Chang, K.-W.; Ringgaard, M.; Lin,
C.-J. Training and Testing Low-Degree Polynomial Data Mappings
via Linear SVM. J. Mach. Learn. Res. 2010, 11, 1471−1490.
(67) Sun, Y.; Seo, J. H.; Takacs, C. J.; Seifter, J.; Heeger, A. J.
Inverted Polymer Solar Cells Integrated with a Low-Temperature-
Annealed Sol-Gel-Derived ZnO Film as an Electron Transport Layer.
Adv. Mater. 2011, 23, 1679−1683.
7444 DOI: 10.1021/acsnano.8b04726
ACS Nano 2018, 12, 7434−7444